Code

What are environment variables, why are they needed, and how to use them

What are environment variables, why are they needed, and how to use them

Free Python Course ➞ Mini-course for beginners and experienced coders. 4 cool portfolio projects, live chat with the speaker. Click and find out what you can learn in the course.

Learn More

I recently prepared material for an online lesson on Linux environment variables. These variables, such as PATH, PS1, and TERM, are familiar to anyone who has ever used the command line. Understanding environment variables is key to working effectively in Linux, as they control system behavior and the user interface. In this lesson, we'll look at how to properly set and use environment variables, which will help you optimize your workflows and increase your productivity on the command line.

Students often ask, "Does anyone really need this today?" The answer is a resounding yes, they do. Environments are a key mechanism for configuring applications on any Unix-like system, whether it's a macOS laptop, a cloud server, or a Kubernetes container. Understanding how to work with environments helps optimize the development process, improve productivity, and ensure proper application configuration. In today's IT world, environment skills are becoming increasingly important for programming and system administration professionals.

The Twelve-Factor App Manifesto, which focuses on building scalable web applications and microservices, emphasizes an important principle: separate configuration from code. Storing settings in environment variables is recommended, providing greater flexibility and simplifying application management across multiple environments. This approach improves security and simplifies the deployment process, making applications more adaptive to change.

Technical lead at Tabby, a fintech company, and a developer of Python testing courses at Skillbox. In my role, I manage teams and projects, ensuring high standards of quality and efficiency. My courses cover key aspects of software testing using Python, providing students with the practical knowledge and skills needed for a successful career in IT. My goal is to make training accessible and effective for anyone wanting to learn Python testing.

In 2000, I first encountered a black screen on FreeBSD 4.0 and immediately fell in love with the Unix console. This experience was a turning point for me, opening up a world of powerful tools and capabilities offered by the Unix operating system. Since then, I've delved deeper into its functionality, significantly expanding my administration and programming skills. The Unix console appeals to me for its simplicity and power, allowing me to effectively manage systems and automate tasks. This experience formed the foundation for my subsequent professional career in information technology.

What are environment variables?

Environment variables are a set of values ​​that determine the settings and behavior of the operating system and the applications running within it. These variables are formed as key-value pairs and stored in memory, which simplifies interaction with programs. Often, articles about environment variables do not pay enough attention to their implementation and practical application. However, understanding how to configure and use environment variables is an important aspect for developers and system administrators. Properly setting environment variables allows you to optimize application performance, improve security and manageability of the system.

It is important to understand that our perception of how technology works is a model in our minds that needs to be refined and adjusted. The most effective way to do this is to refer to the early stages of the technology's development, when its concept was just beginning to take shape and had not yet been saturated with numerous improvements. Let's consider when environment variables were first introduced and how they have evolved to this day. Environment variables, as an important aspect of operating systems, first appeared in 1979 in the seventh version of Unix (Unix V7). This operating system is the basis for many Unix-like systems, such as Linux, macOS, and FreeBSD, playing a key role in their development. One of the first computers to run this version of Unix was the PDP-11. Understanding environment variables is important for developers and system administrators because they allow you to customize the program execution environment and manage processes within the operating system.

The authors of Unix V7, Dennis Ritchie and Ken Thompson, in front of a PDP-11. Photo: Bell Labs

More than forty years ago, the creators of the concept of environment variables sought to solve the problem of software configuration management and interactions between various system components. Environment variables allow developers to configure application settings without changing the code, increasing flexibility and usability. But how relevant is this concept in the modern world of technology? Over time, environment variables have become an integral part of development, especially in the context of cloud computing and containerization, such as Docker. They provide secure and scalable configuration management, making them an essential tool for modern developers. If you're interested in how the environment variable system works and what benefits it provides, let's explore this topic together.

What is the main idea behind environment variables?

Configuration is a key idea that arose to simplify the management of systems and processes. In this context, we can turn to the example of John, a fictitious employee of the DEC laboratory. John demonstrates how configuration helps organize and optimize workflows, ensuring more efficient interaction between the various components of the system. This not only reduces setup time but also improves operational stability. Thus, configuration becomes an essential tool for achieving successful results in the work of modern technologies.

DEC is a company known for its developments in the field of computing, including PDP-11 computers and other devices. Our John had a Teletype Model 33 terminal, which provided a convenient interface for working with data. He launched the executable file of the cat utility simply by entering its name on the terminal. The PDP-11 computer produced the application's output at 100 words per minute, typing line by line like an old-fashioned typewriter. It was noisy but efficient.

Everything worked flawlessly, but progress never stands still. Soon, DEC introduced the VT52 video terminal, and one of them ended up in John's hands.

VT52 video terminal Photo: Columbia University

The new terminal is noticeably different from its predecessor - a display is used instead of a sheet of paper. This change increased the number of characters that could be displayed on the screen. The Teletype Model 33 had a maximum of 72 characters per line, while the VT52 had a maximum of 80 characters. These values ​​couldn't be changed because the Teletype Model 33 functioned as a typewriter, while the VT52 was limited in its character display, supporting only ASCII in a single size.

The version of cat John was using was limited to 72 characters per line. This created the need to develop new versions of the application that supported the VT52. This created a situation where different terminals would require two versions of cat, each differing in just one small setting. This is inefficient, especially as the number of device types could continue to grow. The cat program can't determine which terminal the user is using, which complicates the situation.

The simplest way to explain this situation is to specify a parameter corresponding to the terminal being used when starting the application. For example, for a Teletype Model 33, you would use cat TERM=dumb, while for a new terminal with a monitor, you would use cat TERM=vt52. John discusses this with Dennis Ritchie, and soon a cat command is introduced that supports lines of any length. Now, this command can work on both old and new devices, simply by specifying the required parameter.

Application parameterization allows a single application file to be used in different scenarios. We understand how this works, but the question arises: why is it important to separate configuration into application parameters and environment parameters? Separating these two types of configuration provides greater flexibility and manageability. Application parameters contain settings specific to the application itself, such as business process logic and user interfaces, while environment parameters concern configurations that depend on the runtime environment, such as databases, APIs, and other external services. This separation not only simplifies deployment and testing but also improves security by isolating sensitive data from the main application code.

By the time Unix was introduced, many applications had been developed, and each required integrated support for the TERM parameter. This parameter informed the system of the type of environment being used. However, manually entering TERM each time was inconvenient. The optimal solution would be to configure the environment once and let Unix automatically supply the necessary parameters for all applications. This mechanism, which separated environment parameters from application-specific parameters, was introduced in Unix V7. This significantly simplified the user experience and increased the efficiency of interaction with the system.

Moving some settings into the environment eliminates the need to configure each application individually. This simplifies and makes configuration management more efficient. Using environment variables to store settings helps ensure security and ease of application deployment. This way, developers can focus on creating functionality rather than on routine configuration tasks.

Thanks to John for the clear idea!

Reading the very first manual about the environment

Environment configuration was an important part of working with the operating system in early versions of Unix. The UNIX Time Sharing System — UNIX Programmer's Manual — Seventh Edition, Volume 1A, published in January 1979, contains important information about the shell and its environment. Page 161 of this manual describes the main aspects of environment configuration that were relevant to Unix users at the time. This information provides an overview of how environment management concepts in Unix evolved and how they influenced modern operating systems. Customizing the Unix environment allowed users to tailor the system to their needs, which contributed to increased productivity.

This document presents information on key aspects of the topic. It focuses on important points to consider. The text contains valuable recommendations and practical tips that will help you better understand the issue under consideration. All presented information is structured and clearly presented, making the material easy to digest.

It is important to note that the text emphasizes the significance of the topic, as well as its relevance in the current environment. Potential problems and risks that stakeholders may face are also discussed.

In conclusion, solutions and process optimization techniques are offered, making this material useful for a wide audience.

  • The environment is a list of key-value pairs that is passed to the executed program in the same way as a regular list of arguments.
  • To add environment variables, use the new export command.
  • Commands run from the Shell inherit the same environment.
  • The Shell creates parameters from environment variables.

The system functions similarly to modern Unix-like operating systems. The documentation also mentions environment variables such as PATH, HOME, TERM, and others, which are familiar to many developers. This confirms that the operating principles and configuration remain relevant and applicable in the modern software development environment.

The way of working with environment variables was developed in the early days of Unix and remains relevant to this day. This method allows users to effectively manage system settings and configurations, making it an important tool in software administration and development. Environment variables provide flexibility and convenience, allowing applications to adapt to different conditions and execution environments.

At this point, I realized that this exercise would most likely not be included in the online lesson materials, but I couldn’t stop my “investigation.” I decided to find out where this environment is stored on a Unix system.

Under the Hood

To understand how Unix works with the environment, it’s worth referring to the documentation. The links from the Shell section direct us to ENVIRON (5), which is an important section of the documentation. It describes in detail environment variables and their impact on the execution of processes on Unix. By studying this section, you will better understand how the environment affects system operation and user interaction.

The documentation is a short and easy-to-follow guide. Environment is a global variable created by the EXEC (2) command. Because it is a global variable, it stores data about the environment in process memory. A process in the operating system is defined as a running executable file.

The EXEC call is a mechanism that allows programs to start other processes or commands in the operating system. In doing so, it creates a new execution context in the application's memory, which allows it to manage resources, pass data, and communicate with other processes. In the context of an Environment application, the EXEC call serves as an important function for managing the environment, providing flexibility and the ability to execute commands dynamically. This allows developers to effectively manage processes and optimize application performance.

The EXEC call occurs while the user is running in the Shell. When the user wants to run an application, they enter the path to its executable, and the Shell takes care of the rest. However, the Shell itself does not initiate application startup directly; it simply runs the application. Instead, it calls EXEC, passing the necessary parameters to launch the application and the control parameters of the Unix operating system. This process allows for efficient management of program launches and interaction with the system, ensuring the correct execution of tasks.

Unix handles EXEC through a mechanism that starts a C program by calling the main function with the parameters main(argc, argv, envp). The argc parameter specifies the number of arguments passed to the program, while argv is an array of pointers to these arguments. The envp parameter is a pointer to the environment of the parent process, which in this case is the shell with all its environment variables. This process ensures the correct transfer of data and configuration to the running program, which is a key aspect of Unix systems.

Before executing the main function, a copy of the environment is created, located in the envp variable. This copy is available to the application through the global variable environ. The shell populates the list of variables, while the operating system creates a copy of this list in the new process's memory.

A new process receives a copy of its parent process's environment. This copy allows each application to change its settings without worrying about possible negative consequences for other applications. If a new process starts an application, a child process is created that inherits a copy of the environment only from its parent, excluding its ancestors. This preserves the isolation of settings between different process levels, ensuring the stability and predictability of application operation.

Working with environment variables is supported in Unix, where each application receives its own copy of the parent process's environment in memory. This allows applications to interact with system settings and configurations, providing flexibility and control. Using environment variables allows developers to customize application behavior without having to change the source code, which improves scalability and security.

The concept of an environment was first mentioned in the seventh version of Unix, which confirms the absence of this concept in the sixth version manuals. The documentation for the EXEC (2) call lists only two parameters: argc and argv, while the envp parameter is missing, indicating that the environment did not yet exist at the time. This highlights the evolution of Unix and the importance of environments for modern systems.

In addition to the V7 manual - Annotated Excerpts from the Programmer's Manual - it is mentioned that the familiar Shell (Bourne shell) was developed specifically for the seventh version, replacing the previous version. Without Shell support, the working environment would not be so convenient and functional. This environment provides developers with the necessary tools for efficient execution of tasks and management of system processes, which makes it an important element in the operating system ecosystem.

What happened to the environment later?

Now that we have the answers to all the questions, it is time to turn our attention to the changes in the environment of modern Linux distributions. Linux environment management has undergone significant changes, allowing users to customize their systems more efficiently. New tools and interfaces simplify working with the environment, offering more intuitive configuration and management options. Let's explore how these changes impact users and what new opportunities are opening up for working with Linux.

The test is performed on Linux running in Docker. Tools commonly used by developers are used to solve the problem: the Strace utility allows you to trace system calls, and GDB provides the ability to analyze the contents of process memory. These tools are indispensable for diagnosing and debugging software in the Linux environment.

Before examining system calls, it is important to determine what result I want to achieve. As a starting point, I will choose the exec() call. Why this? Since 1988, POSIX standards have existed that clearly define the structure of system calls. These standards ensure compatibility and predictability of operating systems, which makes them key for developers and engineers. Understanding the exec() call mechanism will provide a deeper understanding of the interaction of applications with the operating system and the optimization of program execution.

Unix V7, which was discussed, was released earlier, but the POSIX standards were largely shaped by Unix-like systems. As a result, Linux can be considered largely POSIX-compliant, allowing us to search for similar calls in that operating system.

This time, I don't need to search for information online because I have access to the man command. It provides documentation on how to use various commands directly in the terminal. This greatly simplifies the process of obtaining the information I need and allows me to focus on completing tasks without having to look at external sources.

Run man exec in an Ubuntu container to open the man page. In Linux, exec is a whole family of system calls, including the environ global variable and passing the environment through the envp pointer. To trace the desired call with strace, you need to know its name—in this case, the execve call. Thanks to the POSIX standards, the mechanisms for working with the environment in Linux are similar to those used in Unix V7.

Now run the cat command in a Bash shell to view the call parameters. This command allows you to display the contents of files in the terminal, which is useful for analysis and data inspection. As a result, you will receive information about how exactly the parameters were passed, which will help you better understand the operation of scripts and commands in the system.

The third argument is a list of variables pointed to by the address 0xaaaadacacb50. To determine the contents of this list, you need to restart the strace utility with the -v switch. This will provide more detailed information about the variables and their values, which will help you better understand the behavior of the program and its interaction with the passed data.

In the third parameter, we see familiar environment variables such as PID, HOSTNAME, HOME, and others. This indicates that this approach is not much different from calling the exec() function in the seventh version of Unix. The main difference is that there is no need to count arguments.

Continuing our investigation, the call descriptions do not indicate the location of environment variables in the memory of the running process. However, this is an interesting aspect, since this is where the very environment we often mention is located. Examining environment variables can provide valuable information about a program's operation and configuration.

I have information about the pointer address from the strace output - 0xaaaadacacb50. Let's investigate what is located at this address in bash memory.

This is a pointer to the start of the bash process environment. We can examine the addresses of these pointers and find that they contain our environment variables. Environment variables play a key role in configuring system and application settings. They can be used to control the behavior of the command shell and pass configuration data to various applications. Studying these variables allows us to better understand how the bash shell works and how to optimize our workflows.

It's time to stop.

Instead of conclusions

Working in the console has become a familiar practice for me, and using environment variables is an important and convenient tool. I was surprised that environment variables have been used in their classic form for such a long time. They help manage application settings and configurations, providing flexibility and ease of development.

The first versions of Unix have fallen into disuse, but they became the basis for many modern operating systems, such as macOS and Linux. To better understand their influence, it is necessary to be curious and refer to old manuals. This will help deepen your knowledge of the structure and operating principles of modern operating systems, as well as learn how to use them effectively.

Thanks to everyone who preserves and publishes documentation of old versions of applications. Thanks to their efforts, I was able to conduct this small study. Preserving historical data about software plays a vital role in understanding its evolution and functionality. These resources allow you to study the changes and improvements that have occurred over time and help developers and users access the information they need.

Learn more about the world of programming in our Telegram channel. Subscribe to stay up-to-date with interesting materials and relevant news!

Rework the text, maintaining the main topic and avoiding unnecessary details. Optimize it for SEO and add necessary information to enhance the content. Avoid emojis and unnecessary symbols, and don't use structured sections like 1, 2, 3, or *. Please provide plain text.

Read also:

  • Basic commands and hotkeys for the Linux terminal (as well as Unix, macOS, and FreeBSD)
  • Test: How well do you know how a processor works?
  • How to use Bash to perform tasks on a computer faster