DevOps – User's blog

In previous chapters, you learned how to store data and what the characteristics of a good web application are. Another important part of the software development process is DevOps – the conductor of the digital kitchen. You already have a working application, an initial MVP that you are continuously developing. Adding new features can become problematic – especially when teamwork is involved. DevOps defines how to properly automate elements of implementation to receive faster feedback on the product, simplify processes, and automate them.

When creating a small application, a significant portion of these elements may seem unnecessary, but as your project grows, you will notice for yourself which elements need to be discarded, changed, or automated – particularly during the deployment and code storage stages.

DevOps, or the collaboration between kitchen staff and waitstaff

To properly understand what DevOps is, we need to explain the acronym:

Dev: project implementation
Ops: application maintenance and deployment

In practice, this means that the entire team is responsible for the application they are building. There are people responsible for writing the code and people whose job is to deploy the application. You may notice that creating a new version of the software comes with certain challenges:

How do you swap files on the server?
Does the new functionality work correctly?
After the changes, is the project ready to launch?

DevOps provides the practices that help answer these questions. Referring back to the kitchen analogy: the cooks prepare the meals, but it is the waitstaff who deliver them, check whether they meet the standards, ensure customers enjoy them, and provide feedback to the cooks.

Another important aspect is the aforementioned analysis of the application’s performance. DevOps involves monitoring the created solution, verifying whether it runs correctly, whether the application responds quickly, doesn’t freeze, and has no availability issues. The most common problem affecting application performance are “bottlenecks”, prevent the application from running efficiently. If a kitchen has only one oven, it cannot bake 12 pizzas at once – it can only make 6. The same analogy applies to web applications: if such a problem occurs somewhere, we need to take action – for example, buy another oven!

Git and GitFlow: One Cookbook for Many Cooks

When developing software, one of the most important elements is the version control system – software that stores the project’s source code, manages changes, and indicates who is responsible for which part of the program. The most popular is Git, used worldwide in business, open-source, and private projects. Even as a one-person team, you can use it, if only to create backups and a history of changes. It’s not easy to find a perfect restaurant analogy, but you can think of it as a recipe journal where previous versions of recipes are recorded along with changes made by individual cooks.

Git is useful when working in large teams; programmers can make changes in parallel without creating unnecessary conflicts. Typically, each person works on a separate branch, and when they finish a task, they merge their solution into the main branch. Another programmer can also review the code before merging – checking its quality, functionality, etc. – this is called a pull request. The programmer sends a request to merge the changes, much like a cook might ask for a dish to be tasted before adding it to the menu.

We also need to mention the workflow organization model. This is especially necessary in huge teams. GitFlow introduces a fixed branch structure, in which each branch has a specific role:

Prod: production code, which is tested and stable
Staging: responsible for testing and fixes before merging into the main branch
Feature: a branch for implementing a new feature, e.g., feature/cooking-robot

This separates development from production, enforces control over the release and deployment process, and also allows programmers to work in parallel. When one finishes a task, it doesn’t affect the other’s work. A cook changing a recipe in their own notebook does not change it in the master cookbook, but also cannot modify the master book directly without the knowledge of other cooks. By using this structure, you can safely create and modify an agent on a separate branch, e.g., feature/cooking-agent.

Unfortunately, this complicates the history and requires training programmers if they haven’t worked in this style before. You might feel that it unnecessarily introduces a large number of branch connections. To put it bluntly, this can be overkill for small projects.

When placing files in a repository, you shouldn’t include everything. Good practice is to include the program code and a list of dependencies. You should not include executable programs or libraries, because anyone can build the program themselves from the code and install the libraries independently. The file that allows you to specify exactly what will be added to the repository is .gitignore. Here you list the files and directories that Git should not synchronize.

Note! Environment variables, such as API keys for language models or database access credentials, should be stored in a separate .env file, not in the code. Additionally, this file should never be tracked by Git, because any changes to it would be visible. This is one way to prevent credential leaks – this issue will be better described in the chapter on security. DevOps is about automating and streamlining processes, but we must not forget the basic elements of security.

CI/CD: An Automated Dish-Release Line

CI/CD represents the first step toward process automation. According to Elon Musk’s golden rules, you must first simplify processes before automating them – here we have simplified the process of introducing changes to the main application. Now it’s time for automation!

The acronym CI stands for Continuous Integration. The idea is that after every change, we can automatically check whether the program meets the standards we’ve set, runs, and is ready for operation. The programmer introduces changes to the repository, and the program automatically verifies whether the code is correct. In restaurant terms, after preparing a new dish, the cook hands it over to a robot that assesses the quality of the dish. If it’s good, it will be added to the menu; if not, the robot tells the cook what’s wrong.

CD, on the other hand, stands for Continuous Delivery. Since we have a stable version of the application ready to run, why not deploy it? Let the machine upload the files to the server, not a human. Manually introducing changes after every significant update is prone to failure, and this process is very easy to automate.

What are the advantages? You gain the most by receiving feedback after introducing changes. If your code were not working correctly, you would only find out during deployment. Code pushed to the repository will also be code that meets appropriate quality standards. This matters if you push changes frequently and the team works in parallel.

GitFlow supports this process – on feature branches, you can run tests, and when merging changes into the develop branch, you can build the changes and deploy them to the test environment. You receive feedback immediately!

In CI/CD, deployment elements must be automated – a human cannot be involved in them. The programmer’s only task is to introduce changes; nothing more – no configuring, no entering additional commands, etc. Slow deployments and testing can discourage programmers from making frequent changes, and the more frequent the changes, the better.

Docker: A Box That Guarantees the Same Taste Everywhere

Imagine a situation: you go to a restaurant in Italy and order the best pizza you’ve ever eaten. You return to Poland, go to an Italian place in the Old Town, and ask for the same pizza, but unfortunately, it doesn’t taste the same. The ingredients were identical, it was baked according to the same recipe, but it’s just not as good.

The same happens with software: one programmer completes a task, introduces changes, and it turns out that the program doesn’t work on the client’s end. This is where containers come to the rescue – a sealed box for the application that contains all the elements needed for it to run. This guarantees that the program will work in any environment – both on the client’s side and on the programmer’s.

The most popular tool for creating containers is Docker. It packages the application into a container during the build stage, creates the appropriate configuration, and saves it in a ready-to-run Docker image. We can run this anywhere in the world!
If only that restaurant in Italy had a box that kept the heat from escaping – you could eat that pizza anywhere in the world!

Docker is used in the CI/CD process: when changes are pushed, a container image with the application is generated, which verifies the program’s correctness and provides the option to run it quickly.

Scaling: How to Serve a Thousand Guests at Once?

Earlier, the issues with application performance were mentioned. Our solution can become overloaded and unable to handle all users. Just as a restaurant has a limited number of seats, every web application has a limit on concurrent users. We need to scale our solution. The two most well-known types of scaling are vertical scaling and horizontal scaling.

Vertical Scaling: Let’s Buy a Bigger Oven!

If the machine has performance problems, we upgrade it. Typically, this means improving the processor and adding more RAM. The better the server’s components, the better it will perform. But this has its limits – we are constrained by budget and technology. Sometimes, we simply cannot get more out of the server at a reasonable price.

Horizontal Scaling: Let’s Buy More Ovens!

Instead of upgrading the machine, we create a second, identical one. We have doubled the resources of our application and can now handle twice as many users, but this also brings its own challenges. We need to set up a guard – a load balancer – which will manage traffic and direct it to the appropriate server. Unfortunately, this adds an additional component and complicates the architecture of our application. However, a well-thought-out architecture allows for virtually infinite horizontal scaling!

Kubernetes: The Maître d’ in Your Culinary Empire

Now that you know what containers are and what horizontal scaling is, you can leverage the powerful tool Kubernetes. Think of it as the restaurant manager who oversees everything happening: how many tables are in the dining area, how many cooks there are, how many ovens the cooks have at their disposal. Kubernetes works similarly – it runs the appropriate applications and ensures there are enough of them. If you say you want three running containers, Kubernetes will make it happen. If it turns out there are too few seats, it will automatically issue a command to add more tables; if there are too many, it will remove some. Kubernetes also handles failures on its own – if a table isn’t working, it gets automatically replaced.

For a small application, Kubernetes might be unnecessary, but as the application grows, managing everything manually becomes unrealistic. These processes need to be automated. Without Kubernetes, someone has to manually respond to failures, monitor server load, and spin up instances when needed. When you see that your MVP has grown into a full-fledged application and the number of users is increasing, consider Kubernetes to help scale your application.

It’s worth noting that Kubernetes works best with an architecture of independent microservices. The ideal scenario is when containers are independent, perform specific tasks, and are interchangeable.

Infrastructure as Code: Digital Blueprints for Building a Restaurant

You are the manager of a restaurant. You need to hire 3 cooks, so you post an ad, conduct interviews, and select the best candidates. Then you need to expand the kitchen, so you go through another tedious process. It’s similar when building an application – you want to create a database, so you do it manually, configure the resources, but you feel that this could be automated.

The solution to this is Infrastructure as Code. There are ready-made tools for configuring your application’s infrastructure without manually clicking through and configuring everything. You write the relevant information in a configuration file, and a program like Terraform or Ansible builds everything automatically. You have an exact architectural blueprint for the restaurant that can be built on any plot of land, complete with all the details. This provides repeatability, change control – you know who changed what and why – and it integrates easily with CI/CD processes.

There are two approaches: declarative (we describe what we want to achieve, e.g., “I want 3 cooks”) or imperative (we describe the steps: “conduct interviews, select 3 cooks”).

Monitoring, or the Health Inspector in Your Restaurant

You’ve introduced a ton of automation into your application – everything runs itself, and you focus on programming. Suddenly, you receive a message from a user saying the application isn’t working. You haven’t checked whether the system is behaving correctly after deployment. An application must not only be built but also monitored – its performance, resource usage. Collect metrics, server status, logs returned by the application, the number of queries per second, and more. Based on this data, take appropriate action – change the infrastructure, fix the application, scale up.

Summary

DevOps is a very broad field that primarily supports and automates processes related to software development. I’ve introduced you to the key concepts, but the most important one is this: the maintenance and creation of an application are a unified process, for which the entire team is responsible. The tools described support this process.

Don’t implement everything at once – do it step by step. A small project certainly doesn’t need Kubernetes, and an MVP doesn’t need a complex GitFlow with CI/CD. You’ll notice that as your solution grows, new elements will naturally become part of its development journey.

A working application is not just code – it’s also about ensuring responsiveness to change and avoiding failures. The entire team is responsible for the restaurant, not just the manager!

Materials

Self-study materials that are related to this topic: