In the last several years, the concept of "platform engineering" has gained considerable traction in enterprise software engineering, ... Hold on a second! It's 2024, we don't need to write this part of the blog post anymore. Let's call on our trusty sidekick Chatty:

Prompt: “Write the introduction to a public tech blog article about the benefits of platform engineering.”

Response: "In the ever-evolving landscape of software development, platform engineering has emerged as a critical component in the successful delivery of software products. Platform engineering, a discipline that focuses on creating and maintaining robust, scalable and efficient software delivery platforms, has become a cornerstone in the world of DevOps and continuous delivery."

"The benefits of platform engineering are manifold. It provides a standardized environment for developers to build, test and deploy their applications, reducing the time and effort required to manage individual development environments. It also fosters collaboration and knowledge sharing among teams, leading to more efficient problem-solving and innovation."

"Moreover, platform engineering promotes the use of best practices and automation, which not only improves the quality and reliability of software but also accelerates its delivery. By abstracting away the complexities of infrastructure management, platform engineering allows developers to focus on what they do best: writing code that solves real-world problems."

Thanks, Chatty. Nice summary. Bonus points for the correct use of "manifold!"

At Addepar, we have been focusing on platform engineering along with the rest of the industry. And as we will see, my team and our cornerstone project recently benefited from these efforts, in all of the ways that Chatty so helpfully outlines for us above.

But first, a little story

The team I am part of at Addepar is responsible for "integrations." This can mean many things, but in our case, it means giving clients access to their financial data in other applications from within Addepar's products, or giving them access to their Addepar data in other applications. An example of the first kind is the ability to jump from a portfolio view in Addepar to the custodian's view of that portfolio. An example of the second kind is the ability for clients to work with their portfolio data in their Salesforce account after Addepar has gathered it from many custodial sources, normalized it, modeled it into complex ownership structures and otherwise enhanced it.

A little more than a year ago, we embarked on the project of creating a workflow platform for building solutions to these problems.

The architectural goals were:

  • Flexibility

    • One software framework for solving many problems

    • Consistent process for deploying and operating that software

  • Reusability

    • Library of tested "workflow functions" that we can reuse in various arrangements as needed

  • Ease of adoption and use

    • Quality documentation and tooling for engineers on the team

    • Software and configuration patterns that are flexible, but point to a "golden path"

    • Simple and repeatable deployment of infrastructure and the application

These goals, being "architectural," encompass the domains of software engineering, DevOps, and of course, the ultimate reason for all of this — meeting the business goals we have to serve our clients.

First deliverables: software engineering and DevOps

Our team had the experience and the chops to make headway on the software engineering side, while keeping an eye on the ultimate value proposition to Addepar and our customers. For our first deliverable, a revamped Salesforce integration that sets the stage for future enhancements, we built reusable libraries, a simple but flexible configuration system, and strong enough tooling, documentation and development patterns to support two new engineers joining the team and the project.

On the DevOps side, the story wasn't all sad trombones, but it definitely wasn't the ecstatic karaoke "Walking on Sunshine" chorus we were singing over on the software side. While the rest of the team focused on the application, I forged ahead with getting the service onto our cloud account. I have experience working in the cloud and building tools to solve specific infra and cloud problems, but I had never built out an entire service from top to bottom. In addition, I was working in an environment for which we had over time created several pieces of the platform engineering puzzle, without them all having come together yet. I spent a lot of time looking for the next piece, or learning whatever was needed to fill in missing pieces, and then sticking it all together with strategically placed bits of bash glue — all while new pieces were being put on the table and others were being pulled off.

The good news: we ended the year on a high note, getting our workflow running correctly, end to end, in our dev environment. The less good news: while our DevOps worked, it was bespoke and some distance from the goals summarized above. This wasn't the foundation for "one model for deploying and operating," nor "simple and repeatable deployment of infrastructure and the application."

Putting it all together: Enter platform engineering

As mentioned earlier, Addepar had been focusing on platform engineering in various ways for the past several years, and there were a lot of useful pieces of the puzzle to pick up. We had developed an internal cloud management platform with standard account, security and authorization features. We had built up an internal community of infra-as-code development, producing a burgeoning collection of infra-as-code modules. And we had taken the opportunity, during a large international expansion project, to revisit our application build and deploy process, incorporating automated tools for deploying and visualizing cloud services.

What was unclear to someone like me, an experienced generalist who needed to stand up a new service, was how to benefit from these somewhat disparate efforts. How to put it all together?

Fortunately, senior leaders wiser than me provided the answer. At the start of the year, I heard about a platform engineering initiative being offered by a new embedded engineering team. The idea sounded perfect to me — we would get an experienced DevOps engineer embedded in our team for a consultative engagement. We bring our use cases, goals and dreams to the table. They bring their knowledge and experience to help us deliver the best solution.

How might we summarize our goals for the engagement? Let's refer back to Chatty's helpful introduction:

  • "... robust, scalable and efficient software delivery platforms ..."

  • "... standardized environment for developers to build, test and deploy their applications ..."

  • "... platform engineering promotes the use of best practices and automation, which not only improves the quality and reliability of software but also accelerates its delivery ..."

  • "... by abstracting away the complexities of infrastructure management, platform engineering allows developers to focus on what they do best ..."

To a great extent, our engagement with embedded engineering helped us realize these goals. I had a clear vision of the infrastructural and, in particular, application deployment attributes we wanted the system to have. I worked with an embedded engineering colleague and another engineer on my team (with our embedded eng friend doing the bulk of the infra work and our engineer focusing on the application deployment pipeline) to replace our infra-as-code with standard layers developed and vendored by our platform team.

Initially, I was hesitant about adopting web-based management platforms for our automation needs, as I have always preferred using command-line tools for automation tasks. However, my perspective changed after seeing our environment depicted as interconnected services within a web interface. This interface offers a comprehensive view, showcasing the health status of services, the sequence of deployment actions, and detailed configurations. It also provides direct access to detailed manifests and logs, which is typically more cumbersome when navigating through various console interfaces. The ability to directly interact with the underlying system state and logs using standard command-line tools remains available as needed.

An implementation success with manifold benefits

Ultimately, we successfully implemented an automated deployment process for our infrastructure and applications, adopting a GitOps approach that integrates seamlessly with our management platform's interface. We further enhanced our setup by creating isolated workflows for our cloud development environment, allowing multiple engineers to develop in parallel. Additionally, we incorporated built-in support for operational metrics, dashboarding, and log analysis, all facilitated through the platform without additional overhead.

These examples cover a lot of the benefits of platform engineering that have Chatty so excited. But what about "... fosters collaboration and knowledge sharing among teams ..."? As one of the early users of embedded engineering expertise, my hope is that we can serve as an example to other teams, and that the impact of Addepar’s platform engineering practice grows over time. All toward the ultimate goal of delivering the most value to our clients: "... by abstracting away the complexities of infrastructure management, platform engineering allows developers to focus on what they do best: writing code that solves real-world problems ...".

Our team is still on the road to success with this project. We have yet to move into closed beta, followed by general production release. All of those chapters in the adventure are yet to come. But what I can say is that at every step along the way, we, and our clients, will benefit from the best that platform engineering at Addepar has to offer.