Architecture

If you're interested in the Distributed Application Runtime (Dapr) and how it can simplify migrating existing workloads to use cloud platform services, then this series might be for you. In this post, we continue modifying the excellent Dapr Traffic Control sample application by swapping the MQTT-based messaging with Azure Storage Queues. We will also look at how you can use the Dapr programming model as an alternative to cloud-provider SDKs.

We're constantly using computer networks and the internet, especially as developers. The Web is the most important application on the Internet; this is the second of two posts aiming to explain the basics of the Web.

If you're interested in the Distributed Application Runtime (Dapr) and how it can simplify migrating existing workloads to use cloud platform services, then this series might be for you. In this post, we continue modifying the excellent Dapr Traffic Control sample application by adding Azure Key Vault integration for its secrets management.

This post focuses on a major application that utilises all of the networking infrastructure described in the previous post, and is often considered (falsely) synonymous with the Internet – the World Wide Web (WWW).

We're constantly using computer networks and the internet, especially as developers. Networking is a large and complex subject - this post aims to covers the basics, aimed at developers.

If you're interested in the Distributed Application Runtime (Dapr) and how it can simplify migrating existing workloads to use cloud platform services, then this series might be for you. In this post, taking the excellent Dapr Traffic Control sample application as our starting point, we look at enabling PaaS Pub-Sub using Azure Service Bus.

If you're interested in the Distributed Application Runtime (Dapr) and how it can simplify migrating existing workloads to use cloud platform services, then this series might be for you. Taking the excellent Dapr Traffic Control sample application as our starting point, we will iteratively migrate components to use Azure cloud platform services.

This blog is part of a series around design patterns. This post focuses on the proxy pattern. The proxy pattern is used to restrict and control access to underlying objects. It is useful in communicating with remote servers, or in adding extra functionality on object access.

This blog is part of a series around design patterns. This post focuses on the facade pattern. The facade pattern provides an interface into a complex system, in order to simplify access to the functionality.

This blog is part of a series around design patterns. This post focuses on the adapter pattern. The adapter pattern is used to enable you to use the functionality defined by a class which is not compatible with the interfaces/architecture of your solution.

This blog is part of a series around design patterns. This post focuses on the decorator pattern. The decorator pattern is used to dynamically add additional properties to objects in order to extend functionality.

Despite the subsequent media reporting, the loss of 16,000 Covid-19 test results at Public Health England wasn't caused by Excel. This post argues that a lack of an appropriate risk and mitigation analysis left the process exposed to human error, which ultimately led to the loss of data and inaccurate reporting. It describes a simple process that could have been applied to prevent the error, and how it will help if you're worried about ensuring quality or reducing risk in your own business, technology or data programmes.

Azure Synapse Link for Cosmos DB is a game-changing piece in the Synapse suite of services - extending the support for SQL on Demand to enable querying over the Cosmos DB Analytical Store. This post explores whether the term 'full stack serverless' should now be extended to cover No-ETL and pay-as-you-query analytics, alongside serverless application architectures.

Often when building software we need to outsource part of the solution to a third party. Common examples include authentication, payment, and email, but there are many others. When first designing a solution the decision around which provider to use can be difficult to tackle. At endjin we have developed a system for answering the question "which provider should I use?".

This blog gives an example of a pattern used for asynchronous initialisation. This pattern allows you to construct readonly objects. It also allows you to use the new nullable references features to ensure your classes are always fully instantiated.

Serverless data architectures enable leaner data insights and operations. How do you reap the rewards while avoiding the potential pitfalls?

Did you know that Azure Synapse has great support for .NET and #csharp? Learning new languages is often a barrier to digital transformation, being able to use existing people, skills, tools and engineering disciplines can be a massive advantage.

Have you or are you about to invest in Azure Databricks? If so, the new Spark offering in Azure Synapse Analytics is likely to have grabbed your attention and rightly so. Why is Microsoft putting yet another Spark offering on the table and what does it mean for you?

Jess and Carmel recently gave a talk at Azure Oxford on "Combatting illegal fishing with Machine Learning and Azure - for less than £10 / month). The recording of that talk is now available for viewing!The talk focuses on the recent work we completed with OceanMind. They run through how to construct a cloud-first architecture based on serverless and data analytics technologies and explore the important principles and challenges in designing this kind of solution. Finally, we see how the architecture we designed through this process not only provides all the benefits of the cloud (reliability, scalability, security), but because of the pay-as-you-go compute model, has a compute cost that we could barely believe!

Wardley Maps are a fantastic tool to help provide situational awareness, in order to help you make better decisions. We use Wardley Maps to help our customers think about the various benefits and trade-offs that can be made when migrating to the Cloud. In this blog post, Jess Panni demonstrates how we used Wardley Maps to plan the migration of OceanMind to Microsoft Azure, and how the maps highlighted where the core value of their platform was, and how PaaS and Serverless services offered the most value for money for the organisation.

The application of scientific experimental process to software development leads to the development of fully-validated solutions. This approach provides you with confidence in designs and means that you can quickly identify ideas which are not worth pursuing. At endjin we use the ideas of hypotheses and experimentation when designing any solution and this gives us full confidence in the designs we produce. In this post we outline the steps and advantages of using this approach.

The Azure CNAB Quickstarts Library helps you get up and running with CNAB and Porter. We've built quickstarts covering solutions like WordPress, Ghost, Mattermost, and data platforms like Apache Airflow, SQL Server AlwaysOn clusters and Kubernetes features like an nginx ingress controller and an Azure AD enabled OAuth2 Proxy. We've condensed all our learnings from the past 9 months of working on the project and turned them into a 10 minute video which explains all the key concepts. We hope this video helps accelerate your own CNAB & Porter epiphanies!

In partnership with Microsoft, we have released Azure CNAB Quickstarts Library on GitHub. CNAB (Cloud-Native Application Bundle) is a new specification designed for facilitating the packaging, installation, upgrading and uninstallation of cloud-native solutions in the cloud, on-premise or on the edge. We've created a number of quickstarts covering Apache Airflow, Azure Kubernetes Service, Ghost, Kubeflow, SQL Server Always On and Wordpress to help demonstrate the power of CNAB and Porter.

In this post we show how a combination of Kubernetes, Azure Durable Functions and Azure API Management can be used to make legacy batch processing code available as a RESTful API. This is a great example of how serverless technologies can be used to expose legacy software to the public internet in a controlled way, allowing you to reap some of the benefits of a cloud first approach without fully rewriting and migrating existing software.

In this post, Ian describes some of the highlights from the NDC London conference

With a variety of integration support through client SDKs, PowerShell cmdlets and REST APIs, it can be hard to know where to start with integrating Azure Analysis Services into your custom applications. This posts walks through the options, and lays out a simple guide to choosing the right framework.

In this blog from the Azure Advent Calendar 2019 we discuss building a secure data solution using Azure Data Lake. Data Lake has many features which enable fine grained security and data separation. It is also built on Azure Storage which enables us to take advantage of all of those features and means that ADLS is still a cost effective storage option!This post runs through some of the great features of ADLS and runs through an example of how we build our solutions using this technology!

Expression Trees are one of .NET's super powers; but they are hugely overlooked and misunderstood. They can be used for on-the-fly code generation; building DSLs, and producing high performance computational algorithms. I'm trying to gather evidence about how you are using them, to make a case for future investments in the Expression Tree subsystem in .NET Core / .NET 5.0

While on first inspection Azure Function look like a good candidate for long running operations as they can run for 10 minutes on a Consumption plan, HTTP Triggers only run for 230 seconds because of a limitation with the Azure Load Balancer. In this blog post Jess Panni demonstrates how Durable Functions can be used instead.

This blog is part of a series around design patterns. This post focuses on the composite pattern. The composite pattern is often used in situations where you want to me able to treat groups and individuals in the same way during processing.

Building a secure solution on Azure can be a daunting task. Using Azure Functions and Managed Identities, we have built up a pattern for giving services access to one another, woithout the need to store credentials. These managed identities can be given access to necessary resources. For example, they can be granted roles and added to access control lists in ADLS Gen2 accounts, or the ability to access keys in key vault. This means that data can be securely accessed without needing to store connection strings or app passwords.

This is the first blog in a series about design patterns. This blog focuses on the differences between the factory method and abstract factory patterns. The factory method is a method which takes the creation of objects and moves it out of the main body of the code. An abstract factory is similar to the factory method, but instead of a method it is an object in its own right.

Software architecture and Agile project planning are often seen to be at odds. However, here at endjin we think that the way in which they intersect solves a lot of the common issues surrounding architecture. The key to successful architecture is constantly keeping the drivers in mind and having a tight communication loop as the architecture is implemented. These concepts are ones which are key to agile project management, and the combination of these two disciplines can be extremely powerful and successful when applied correctly.

Creating a architecture in an iterative and incremental solution delivery is challenging. I've evolved an approach akin to Painting By Numbers, and am inspired by a quote from the artist and film maker Dave McKean.

At endjin we have a high quality bar when it comes to our code. As part of this we carry out regular code reviews. One of the tools we have used for these code reviews is NDepend. This is the second in a blog series written as we carried out that process. This post focuses on the insight you can quickly gain just by glancing at the NDepend UI.

At endjin we have a high quality bar when it comes to our code. As part of this we carry out regular code reviews. One of the tools we have used for these code reviews is NDepend. This is the first in a blog series written as we carried out that process. This post runs through the different metrics used by NDepend, and the reasons that each of these can be an indication of code quality.

This is the final blog in a series which delves into how the Rx operators work under the covers. This series aims to provide a greater understanding of Rx and its operators. This post focuses on the JOIN operator.

This post walks through the fix for DLL locking errors when trying to deploy an Azure Function. The solution was to switch over to the new "deploy from package" option when deploying the functions. This fixes the file locking problem because instead of deploying the DLLs, the function will run from a package file added to its directory.

This is the fourth blog in a series which delves into how the Rx operators work under the covers. This series aims to provide a greater understanding of Rx and its operators. This post focuses on the WINDOW operator.

When he joined endjin, Technical Fellow Ian sat down with founder Howard for a Q&A session. This was originally published on LinkedIn in 5 parts, but is republished here, in full. Ian talks about his path into computing, some highlights of his career, the evolution of the .NET ecosystem, AI, and the software engineering life.

This is the first blog in a series which delves into how the Rx operators work under the covers. This series aims to provide a greater understanding of Rx and its operators. This post focuses on the WHERE operator.

This is the second blog in a series about data flow. This post delves into TPL dataflow.The task parallel library is a .NET library which aims to make parallel processing and concurrency simpler to work with. The TPL dataflow library is specifically aimed at making parallel data processing more understandable via a pipeline-based model.

This is the first blog in a series about dataflow. The series focuses on TPL dataflow, but this post gives an overview of dataflow as a whole.The crucial thing to understand when using dataflow is that the data is in control. In most conventional programming languages, the programmer determines how and when the code will run. In dataflow, it is the data that drives how the program executes. The movement of data controls the flow of the program.

This is a post about the importance of reducing your dependencies as much as possible when designing solutions.If as much as possible is under your own control, there is far less room for synchronization errors and update conflicts.

Ian Griffiths recently joined endjin as a Technical Fellow. We had a long fireside chat, which has been broken down into a 5 part series covering .NET, the Cloud, AI, ML, teaching software engineering, and why he joined endjin.

Rest APIs underpin many Digital Transformation strategies. As with all software projects the devil is in the detail; your API strategy, governance, security, design principals are as, if not more imporant than building a working API.

We've been helping customers adopt Microsoft Azure since 2010, we have produced a lot of thought leadership to help people think about the steps required, the risk involved and how to plan a successful adoption.

We've just released an updated version of our Azure Technology Selector poster - helping you to choose the right Azure service, based on your use-case.

Hymans Robertson was set up in Glasgow in 1921 and is one of the longest established independent firms of consultants and actuaries in the UK. Hymans Robertson soon realised that the computational requirements of their models exceeded the capacity of their on-premise datacentres and that the most cost effective solution would be to use the cloud to perform their Big Compute. But before they could harness the cloud to help them solve their Big Data problems, the business needed to understand the ramifications of moving to the cloud; everything from regulatory, risk and compliance concerns,  to how their internal Ops team would need to evolve and adapt, and how to deal with moving data from on-prem into the cloud.

An experiment to examine if we can create a tool that can automatically detect accidental breaking changes in a .NET API and fail the build, so we can properly SemVer our NuGet packages.

Consumption based pricing is a one of the USPs of Cloud PaaS services, but the default settings aren't necessarily optimised for cost. Significant savings can be made from understanding your workload.