Note to reader: Time stamps embedded in the text below in [ ] are there to help you navigate to the related section of the full interview video included in this blog.
Maintaining container solutions require the management of secrets across different platforms. Of course, a secret has to remain secret no matter where it is stored. The External Secrets Operator (ESO) project is an open-source solution that has come together across the Kubernetes ecosystem to create one shared resource for integrating external secret management systems.
Carla Gaggini and Gustavo Carvalho are consultants for Container Solutions in London, and they were seeing the same pattern across their customer base. At the same time more than a dozen projects had emerged to tackle the management of external secrets across different organizations [4:57]. The two of them got 15 people together on a call and what emerged was ESO, one project to bring together all the coders and code and create one open source project for secrets management across the Kubernetes ecosystem–which includes a lot of different infrastructure projects.
What I loved about their story was the way in which it demonstrates the positive attitude of sharing and contribution within the open source community. It’s also a perfect example of how innovation is increasingly coming from open source models, a kind of “crowdsourcing” of knowledge, rather than from the big corporations. Of course the corporations are always part of the mix, providing resources as well as benefiting from open source, but this story definitely demonstrates my favorite part of Kubecon: the fantastic friendships and cooperation that are formed within the community.
So what is ESO all about?
The External Secrets Operator (ESO) solves a fairly straighforward problem: passing secret data from its original storage location and converting it to a Kubernetes secrets. For those who are less technical, a secret is exactly what it sounds like: any type of data that needs to be kept secret. That could be a password, token, or key. Typically the term refers to a piece of data that is used to secure accounts and data. Basically, the ESO does one thing: it reads information from external APIs and automatically injects the data into a Kubernetes Secret.
- The task seems so striaghtforward that many developers were developing their own solutions, but by joining forces the ESO project has been able to accomplish several things.
- First of all, everyone saves time by eliminating duplicated effort.
- Secondly, as I alluded to above, people make friends. Carla and Gustavo were overwhelmed by the number of enthusiasts who came to their booth because ESO has solved real problems for them.
- Thirdly, ESO brings together API hooks for the myriad of interfaces from cloud providers. The big ones are AWS Secrets Manager, HashiCorp Vault, Google Secrets Manager, and Azure Key Vault, but there are plenty of other storage sources for secrets. No one coder could manage all that, but as a consolidated project, ESO brings together contributors from all of the different communities.
- Finally, by coming together, the project creates more robust code. When it comes to secrets or anything having to do with security, more participants means more ideas on how to keep things secure in transmission and more eyes on the code to make sure bugs don’t sneak in.
One of the innovations of ESO is that it federates the authentication so that no sensitive information needs to be stored in the cluster [10:52]. The project team has created numerous configuration capabilities to determine how to store the secrets. The system carefully stores the secrets separate from the information that is not sensitive, so they are able to offer autonomy and flexibility to develop their applications around the secrets without being worried about exposing secret information.
The not-so-secret roadmap
As with other open source projects, ESO is transparent and Gustavo was happy to share what’s in the pipeline for ESO [13:09]. ESO already allows developer to cluster secret stores, and extend them or bind them with role-based access controls and other tools such as gatekeepers.
The version ESO is working on is almost feature-complete for validation web hooks and conversion web hooks. He mentioned there are only two features to make it complete, plus bug fixing and testing the code before it can be called general availability.
Sync of the secrets has been a major feature that the team has been focusing on, with two full-time developers working specifically on that feature set.
The original specification was for 4 provider interfaces, but ESO already supports 15 providers, and has a few more requests in the pipeline. Community collaboration has made it possible to answer the needs as they emerge [14:54]. And as with any security-oriented project, the solution makes sure that it’s compliant and secure at all times. The approach has adopted the Kubernetes approach to extensibility, making it accessible to any community member who wants to use the tools provided by ESO.
The pros and cons of the Kubernetes ecosystem
Being part of an extensible ecosystem has both pros and cons, as we discussed. On the one hand, Kubernetes has a very open policy in terms of what can be built by the participants in the community. This allows for projects such as ESO, which bundles certain types of operators together because there are no particular specifications or limitations on doing so. On the other hand, these types of necessary infrastructure projects do end up falling on innovators within the community such as Carla and Gustavo. In other words, there’s a lot of flexibility, but there also tend to be gaps in the product that spawn these types of initiatives.
One of the points we made was that there seem to be some core components that are missing from Kubernetes, but it’s hard to define what they are. The Kubernetes project has a looser definition of the core than many other open source projects. But, as Gustavo pointed out, there’s a tradeoff. Having an open project allowed people to mold their own solutions and may have potentially resulted in a wider adoption of Kubernetes.
As with any open source project, Kubernetes started with a small group of founders and there wasn’t a community to set up the core values and project definition. At this point in maturity of the project, Gustavo pointed out, it’s probably time for the community to come together on a set of values and project definitions for Kubernetes.
Security isn’t sexy
One thing that was obvious in Kubecon was that the security stack is shaping up within the ecosystem. The ESO team expressed enthusiasm about the other projects that are also emerging in terms of Kubernetes security. While security can seem like a boring topic, it’s obviously a cornerstone of any project today. Kubernetes has come a long way in terms of the security solutions supporting the ecosystem. ESO is a part of that security landscape that has experienced tremendous excitement and growth of the community contributors.
We’d love to hear what you think. Ask a question or leave a comment below.
And stay connected with Cisco DevNet on social!
LinkedIn | Twitter @CiscoDevNet | Facebook | Developer Video Channel
CONNECT WITH US