Why should you consider ConfigSeeder?
Over the past few years we have worked for different clients in different industries. We haven't been to a single company that hasn't had problems managing configuration data. In this article, we describe 10 of the most frequent problems companies have with the management of configuration data and show, how using an application configuration management tool like ConfigSeeder® can help make the management of application configuration data easier and faster.
Often seen scenarios:
- Distributed configuration
- Configuration in different formats
- Passwords stored in Source Control
- Configuration changes must be done immediately before a release
- What is a Keystore?!?
- Access to Kubernetes Secrets for developers
- Prevent media breaks
- Configuration Changes performed by the Application and Release Management
- History of configuration data and restore of a change
- Compare configuration data before doing a release
Get more information about how ConfigSeeder® works and have a look at our features to see what ConfigSeeder® can do.
Problem definitions and pain points
Distributed configuration
The configuration is spread around over multiple servers and/or different source code repositories. Perhaps only the application owner knows exactly which service and/or application server needs which configuration data. Where and in which format the certificate and private keys must be stored. Perhaps this person doesn’t even work for you anymore. For the daily business, this can be OK until the day arrives when the service stops working. Then some questions will be asked:
- Who can access the VM on which the service is deployed?
- Who knows the details about how to configure the service?
Even more problematic is this scenario if you don’t run the VMs yourself but have sourced-out operations to another company. We saw cases where answering the request for providing the configuration of an application took up to two weeks.
With ConfigSeeder, you can manage the configuration of your service and also other infrastructure like application servers in one place. You have different ways to access the configuration data stored in ConfigSeeder, for example:
- Access the required configuration data at run time with your application
- Provide the configuration in the form of files to your application
- Provide the configuration in the form of Kubernetes Objects to your application
Configuration in different formats
Nearly every company we have looked into uses more than one technology. If you heavily use Java, you probably develop your applications using Spring Boot. Most of your applications could run containerized in Kubernetes. Some of them might (still) be deployed on a standalone Tomcat or even a Weblogic or WebSphere application server. Let’s have a look at the format of the required configuration files:
- Spring Boot: properties or yaml
- Kubernetes (deployment descriptors for your applications): yaml or json
- Tomcat, Weblogic, WebSphere: xml
There could also be configuration files with other content:
- Some of your applications (maybe a 3rd party application) might require a license file with binary content.
- If you do TLS termination in your applications, you might need key and trust stores.
With ConfigSeeder, you can manage all the mentioned configuration files centralized at one location. ConfigSeeder can automatically generate configuration files in property, JSON, and YAML format that contains all stored configuration values. If you need finer control, you can create configuration files with any text-based content with our templating mechanism.
Passwords stored in Source Control
Passwords stored in plain text in a source control system like git … we all know that this is a taboo, and nearly in every company, this is still a problem. Of course, you can try to restrict the access to the git repositories, but the problem remains. With newer technologies like Kubernetes there exist tools like sealed secrets to store passwords in an encrypted way so the developers can’t see the passwords.
We give security high priority. Therefore ConfigSeeder encrypts all your data by default. It will also show Passwords and other data marked as sensitive only as *** per default. If you have sufficient access permissions, ConfigSeeder will allow you to see passwords in plain text. Depending on your use case, ConfigSeeder can store passwords and other sensitive data automatically as Kubernetes Secrets or you can look them up out of your application with the clients we provide.
Configuration changes must be done immediately before a release
Sometimes a company already uses a configuration management system, but this system doesn’t support versions or releases. Meaning that if a configuration value needs to be changed for the next release, the change can’t be done before the next release is installed (because the current release can’t work with the new value of the configuration parameter).
With ConfigSeeder you can define in which date and or version range a configuration value is valid. This means that your developers (or release- or application managers) can update the configuration when they implement a change and flag the new configuration with the correct date or version. The configuration will be available when you install the new release or when the application requests the configuration data. With this feature alone, one of our customers was able to reduce the number of required application patches by ~33%.
What is a Keystore?!?
TLS based protocols like https are becoming more and more important. As a result, your engineers need to provide certificates and private keys to your applications and systems. But many developers don’t like to deal with security-related issues. They may not know how to create a key or trust store. Even some system engineers don’t know that much about certificates. We have seen cases where requesting a key and trust store took around a month and the created trust store contained the wrong certificates.
In conclusion, you probably don’t want to create key and trust stores yourselves. With ConfigSeeder you can automate this task. You just store your certificates and private keys in ConfigSeeder and define which certificates and/or private keys a key or trust store should contain. ConfigSeeder (OS Connector or Kubernetes Connector) will provide the defined key and trust stores to the target runtime environment.
Access to Kubernetes Secrets for Developers
A company uses Kubernetes (or Rancher or OpenShift) and the developers should work in a DevOps way. They deploy their application themselves and are responsible to do all the configuration work, but, the developers don’t have access to secrets. (The credentials for the DB, critical service and some private keys are stored in the secrets. The developers shouldn’t have access to these credentials). The consequence of the decision to to not grant access to the secrets is that somebody else, probably the Kubernetes engineers, has to manage all the secrets for all applications and all environments. If all clusters are managed by one tool like Rancher, all secrets can be created at one location. Otherwise, the secrets have to be managed at different locations. On the other hand, the developers also won’t be able to see how the secrets created by the engineers look like.
With ConfigSeeder, you can give the developers permission to prepare the configuration data required for creating the secrets. The Kubernetes Connector will then automatically create the secrets. Depending on your security requirements, the developers can prepare the data for all environments or perhaps all test environments. The Kubernetes engineers can then prepare the data for production. All this configuration data will be stored centralized in one location, the ConfigSeeder.
Prevent media breaks
In a lot of companies we see the following scenario:
- The developer orders new DB credentials for the environments DEV, TEST, and PROD.
- DBA creates the accounts, sends the credentials via two channels to the developer (username in the ticketing system, password sometimes per encrypted zip or just in an email)
- For the DEV and perhaps TEST environment, the developer can configure the required configuration (username, password, connection string) himself.
- For the PROD environment, he has to send the configuration data (normally username and password through different channels) to a system engineer or application or release manager.
- The engineer or manager than configures the application without having a context about the changes.
- If your configuration management tool doesn’t support versioning configuration data (see Configuration changes must be done immediately before a release), you have to remember to update the configuration at the right time. Otherwise, the new release will fail.
This process contains some media breaks, and every media break increases the likelihood of someone making a mistake. For every x credentials or other configuration data ordered, something won’t work and error analysis needs to be done.
With ConfigSeeder, you can reduce the number of media breaks. You can grant the (lead) developer the permissions to store data in ConfigSeeder himself. The developer doesn’t need access to the runtime environment (like Kubernetes or a VM), because ConfigSeeder and its connectors will distribute the configuration changes. If the developer can’t have access to some environments, perhaps you can give the DBAs permission to update the configuration themselves?
Configuration Changes done by the Application & Release Management
There are reasons that application or release managers should be responsible for configuring at least the sensitive data of your applications (e.g. separation of concern, the developers shouldn’t be able to access the applications and databases on production, regulatory requirements, …). If this is the case, the application and release managers need the tools to do these configuration changes. And let’s be realistic, git is a tool from developers for developers, you can’t really expect an application or release manager to configure some dozen applications in as many git repositories. And of course, if the developer shouldn’t know credentials it doesn’t make sense for somebody else to store this data in the repository the developer can access (Yes there are tools like sealed secrets, but the bring there own challenges).
With ConfigSeeder, you can easily grant very specific permissions to users with different roles, for example:
- Developers can read and write all configuration data for non-prod environments. On production, they can only read configuration data. Depending on your requirements, you can grant the developers permission to read sensitive data on specific environments but not on production.
- Application and release managers can edit all configuration data on all environments.
In contrast to a source control system like git, ConfigSeeder is a tool that can easily be used by your application and release managers.
History of configuration data and restore of a change
One of our customers had a tool for centralized configuration management in place. This system had a history function implemented, but only for the latest change and the change had to be done on the same day. It also was not possible to restore a changed value. Depending on which tool you use, you might have similar limitations. When you store your configuration data in a source control system, of course, you have the complete history and can restore old values. But probably only the developers have the ability to do that (also see Configuration Changes done by the Application & Release Management). If you manage configuration data directly in Kubernetes or on the target VM, you probably don’t really have a history.
With ConfigSeeder, you can tag all configuration values with a value. Even if you don’t use this feature, ConfigSeeder will log every configuration change and allow you to look into the history of a configuration value. You can restore a configuration value by selecting the desired value and reactivate it with a simple click. There is no technical hurdle.
Compare configuration data before doing a release
Imagine that you have a service that requires a new configuration value for the next release. So the developer adds the new value for the TEST environment develops the change, tests the service against the TEST configuration, and installs the new release. Your tester verifies the new functionality, finds a bug and assigns the change back to the developer. The developer fixes the bug, reinstalls the release on TEST … if the change is big enough, this can take quite a while. At some point, the tester OKs the change, and the Release Management installs the release on PROD. Boom! The new release requires a new configuration value but nobody has done the change on PROD.
This shouldn’t happen with ConfigSeeder because it is very easy to compare the configuration data of two environments. If you see a configuration value which exists for all environment but PROD, you can fix the invalid configuration before you install the release. With ConfigSeeder you can compare configuration data of multiple environments, versions, dates, or contexts.