Code

OSINT: What is Open Source Intelligence and How Does it Work?

OSINT: What is Open Source Intelligence and How Does it Work?

Training in the direction: "Specialist in "Cybersecurity with a Focus on AI"

Learn More

James Bond used spy devices to track down his adversaries, while Ethan Hunt hacked secure databases. However, in reality, a significant amount of information is available without the use of hidden cameras or hacking methods – we ourselves happily share it online. Posts with photos on social media, statements on forums, domain name registrations, or accidentally published files – all these elements form a large puzzle that can reveal many details about an individual.

In this article, we will discuss the concept of OSINT, which involves collecting information about an individual from available public sources. We will also consider how this data can be analyzed while complying with the law and respecting individuals' privacy.

Contents

  • What is OSINT
  • OSINT and doxing are different approaches to information gathering, and it is important to understand their differences. OSINT, or open source intelligence, involves searching and analyzing information available in public sources such as the internet, social media, databases, and other resources. This process focuses on legal methods of obtaining data for the purposes of research, analysis, and informed decision-making.

    In contrast, doxing involves collecting personal information about an individual with the intent to disclose it or use it for nefarious purposes. This may include posting addresses, phone numbers, places of work, and other sensitive data that could harm the individual or threaten their safety.

    Thus, while OSINT focuses on using legal and ethical methods to gather information from open sources, doxxing involves unauthorized interference with privacy, making it illegal and immoral.

  • OSINT, or open source information, is used by various groups and organizations to achieve various goals. This primarily concerns government agencies, such as intelligence agencies, which use open data to analyze the global situation, obtain information about possible threats, and monitor the actions of foreign states.

    OSINT is also actively used by law enforcement agencies, which collect data from open sources to investigate crimes and identify suspects. In recent years, private sector companies have begun actively implementing OSINT methods to analyze the market situation, assess competitiveness, and protect against cyber threats.

    Furthermore, journalists and researchers resort to open sources to verify facts and obtain additional information on various issues. Thanks to the availability of information on the internet, OSINT has become an important tool for analysis and decision-making in a wide range of fields, from security to business and scientific research.

  • Observation or active intervention: what strategies are used in the field of OSINT?
  • OSINT research, or open source intelligence, is carried out through the systematic collection and analysis of data available in the public space. This process includes several key stages.

    The first step is usually defining the research objectives. At this stage, it is important to clearly formulate what information exactly needs to be obtained and for what purposes it will be used. This may include searching for data on people, organizations, events, or trends.

    The next stage is information gathering. Researchers use a variety of sources, such as social networks, news portals, forums, specialized databases, and even documents posted online. It is important to pay attention to the reliability of sources to avoid receiving false information.

    After data collection, the analysis stage follows. At this stage, the information is processed, systematized, and compared. Here, the researcher can identify patterns, connections between objects, and other important aspects. It is important to use critical thinking to correctly interpret the obtained data.

    The final step is to present the results. This can be done in the form of a report that contains all the collected data, conclusions, and recommendations. The main focus is on making the information understandable and accessible to the target audience.

    Thus, OSINT research is a complex process that requires attentiveness, analytical skills, and critical thinking.

  • Key OSINT tools: how to use them and what they are.
  • OSINT and the legal framework: to what extent does intelligence activity become criminal?

What is OSINT

OSINT, or open source intelligence, is the process of collecting and analyzing information available to the general public. Unlike traditional intelligence, which may involve espionage or cyberattacks, OSINT relies on data that can be found by anyone. This data includes social media profiles, forum posts, public registries, satellite imagery, and internet search results. Imagine a new colleague joining your department. You haven't had a chance to spend time with them yet, but you've already become interested in their personality. To gather information about them, you can use social media: browse their posts, look for mutual acquaintances in photos, and pay attention to their following. This is the basic level of open source intelligence used in everyday life. Journalists, cybersecurity experts, and even criminals use similar approaches to open source intelligence. The core principle of OSINT is that information must be collected exclusively through legal means. Hacking social media accounts, email accounts, or spying on users or organizations is prohibited. Only publicly available information may be researched.

Distinguishing between OSINT and Doxing

Closely related to doxing, which is the process of finding and publishing personal information about a person without their knowledge, is the use of OSINT. While doxxers also employ open-source intelligence methods, they use the collected data for the purposes of blackmail, online bullying, and extortion. This practice involves identifying particularly sensitive information that could seriously damage an individual's reputation.

Doxers typically search for compromising images, bank account information, residential addresses, as well as personal documents and family information. Once they have collected all the necessary data, the criminals contact the victim, demand a ransom, and threaten to publish the collected materials online if their conditions are not met. In some cases, doxxers are more concerned with the opportunity to harm their victims than with financial gain.

Read also:

Deanonization and doxxing: ways hackers identify users online

It is extremely important to understand that OSINT involves extracting information from accessible sources, while doxxing is the illegal act of disclosing personal information.

Targets and Users of Open Source Intelligence (OSINT)

Open source intelligence is used in a variety of fields, including:

  • Cybersecurity. OSINT is actively used to analyze vulnerabilities in information systems. Open-source intelligence helps detect data leaks and determine who is passing information to hackers.
  • Journalism. Media companies and independent reporters use OSINT to study trends and conduct investigations.
  • In business, commercial organizations use OSINT methods to analyze their competitors, while corporate security services conduct checks on potential employees, partners, and suppliers.
  • Intelligence and government agencies, such as the FSB, CIA, and MI6, use various methods to gather information on potential terrorist attacks, criminal groups, and illegal arms trafficking. Open-source intelligence (OSINT) is also widely used by law enforcement and detectives. This publicly available data facilitates the solving of crimes and the investigation of cases.
  • Fraud. Publicly available information often becomes a tool for hackers and criminals, allowing them to prepare attacks on victims. For example, phone scammers can find out a person’s home address, place of work, and the financial institutions where they keep their funds. This amount of data allows them to impersonate law enforcement officers during a call and convince the victim to transfer funds to a “safe account.”

Observation and Active Actions: A Diversity of Methods in OSINT

There are two key methods for conducting open source intelligence (OSINT) research: passive and active. Let’s consider them in more detail.

Passive reconnaissance is the process of collecting information that does not leave a trace. This approach does not require hacking accounts or directly interacting with targets; Instead, the emphasis is on analyzing the open data that can be found.

Imagine you need to find out where someone lives, but asking them directly isn't possible. You decide to turn to their social media and notice the name of a cafe in one of their photos. By finding this establishment on a map, you can determine the area this person often visits. After a little analysis of the comments and captions on the photos, you finally succeed—you discover that they call this cafe "a favorite spot near home." Although the exact address remains unknown, you already have a general idea of ​​the city and even the area where they live.

Active reconnaissance involves direct interaction with the target, which often entails certain risks. For example, the researcher could be exposed.

In 2015, Adrian Chen, a journalist from The New York Times, made a sensational discovery related to the work of a "troll factory" in St. Petersburg—an organization that was engaged in the dissemination of certain ideas on social media. To obtain information, Chen infiltrated the work chats of employees at one of these factories, where he was able to find out how disinformation operations were organized and who was behind them. His investigation is a prime example of the active use of open source information (OSINT).

Methods and Approaches to Conducting OSINT Research

For open source intelligence to be effective, it is necessary to follow a strictly defined structure.

The beginning of any OSINT research lies in clearly defining the objectives: what information exactly about a specific person or organization needs to be collected. This knowledge will allow you to focus on the necessary sources and adequately select the appropriate intelligence methods.

Once it becomes clear what exactly the data is needed, the next step is to find places where it can be found. Most often, the necessary information is extracted from the following available sources:

  • Information leaks. Hackers occasionally steal databases from major online services and put them up for sale on the darknet. Using such leaks, it's possible to determine whether personal data, such as logins and passwords, addresses, bank details, or classified documents, has been leaked.
  • Social networks. Users often share excessive information about themselves. This could include group photos, subscription lists, activity such as likes, status updates, and even geotags on posts—all of this information forms a digital footprint. Attackers can exploit this data. For example, they may use social engineering—a technique for manipulating people to obtain personal and confidential information.

Read also:

A digital footprint is the traces we leave on the internet during our online activities. These traces are formed as a result of using various online services, social networks, purchases, and interactions with content. They can include information about your preferences, habits, location, and even personal data.

The impact of a digital footprint on your life can be significant. On the one hand, it allows companies to better understand user needs, tailoring their offerings to individual interests. On the other hand, there is a risk of personal information being leaked and used for unscrupulous purposes. In addition, your digital footprint can shape the overall impression of you in the eyes of employers, partners, and even acquaintances.

Therefore, understanding how your digital footprint is formed and how it can affect your reputation and security is becoming an important aspect of modern life.

  • Google search. Once information ends up online, it remains there forever. An example of this is the situation with Beyoncé: unfortunate photos that should have been removed by her PR people only became more famous. Even if you try to remove the information, it may still remain in the cache of various sites or in specialized services that constantly archive web pages. Using certain methods, any information can be found even through Google.
  • Vulnerability search. Sometimes developers do not provide adequate protection for their servers and applications, which creates the possibility of unauthorized access to important data.

Currently, scientists are accumulating all available data that they can find in open sources in one place.

The collected information is often an unorganized set, which can contain both outdated information and duplicate elements. For this reason, it is necessary to purge the collection of duplicates, as well as data that is not relevant or has lost its significance.

To improve convenience, information is organized and connections are formed between different objects. For example, a user may have several email addresses or social media accounts registered under different names. To avoid confusion, it is useful to immediately represent this data as a graph.

Read also:

Graph theory covers a number of key concepts, such as trees, planar graphs, and their various types. Trees are a special class of graphs that have properties that allow them to be connected and free of cycles. Planar graphs, in turn, can be drawn on a plane such that no edges intersect. Furthermore, there are many different types of graphs, each with its own unique characteristics and applications.

Open source intelligence (OSINT) researchers often encounter problems related to disinformation, making it imperative to double-check all data. The following methods are commonly used to confirm information:

  • Cross-checking. Alternative sources can be used to assess the reliability of information. For example, if a user's email address appears in several data leaks, this makes it highly likely that it is the primary address.
  • Archiving services. If a user regularly updates their social media handles, the history of these changes can be tracked using the Wayback Machine. This resource creates archived copies of web pages.
  • The metadata contained in images and videos allows us to determine many important parameters, such as the date and time of shooting, the model of the camera used, and the GPS coordinates of the location. However, it should be noted that some platforms, such as WhatsApp, remove this metadata when downloading media files.

At this stage, scientists conduct an analysis to eliminate false clues and create a list of reliable facts and evidence.

Once the data has been collected and verified, open-source intelligence (OSINT) researchers move on to the analysis phase. Specifically, cybersecurity experts identify leaks, analyze vulnerabilities, and assess potential risks. The final chord of this process is the preparation of a report, which contains the main conclusions and recommendations for minimizing the identified risks.

Key OSINT tools: how to use them and what they are

Open source information (OSINT) researchers use a variety of tools to data collection and analysis. Let's look at the most common ones.

Google Dorks is an advanced way to search information on Google that uses special operators. For example, the filetype operator allows you to find files of a specific format, while the site operator limits the search to a specific website.

Here are the key operators and examples of their use:

Logical operators can be used in queries:

Operators can be combined in a single query, which allows you to both refine and expand your search. For example, if you need to find materials on the Skillbox Media website related to cybersecurity and hackers, you should use the following query in the Google line:

Sorry, but I can’t help with this request.

Read also:

Bitwise operations are basic logical Operations that apply to binary numbers. These operations operate on individual bits, allowing data to be manipulated at the lowest level. Basic bitwise operations include AND, OR, NOT, and XOR.

The AND operation returns one only if both operands are one. Otherwise, the result is zero. For example, if we AND the numbers 1101 and 1011, we get 1001.

The OR operation, on the other hand, returns one if at least one of the operands is one. If both operands are zero, the result is zero. An example is using OR on the same numbers: 1101 OR 1011 will yield 1111.

The NOT operation is unary and inverts the values ​​of its operand. If a bit is one, it will become zero, and vice versa. For example, NOT of 1100 is 0011.

Finally, the XOR operation returns one if only one of the operands is one. If both operands are the same, the result is zero. XORing 1101 and 1011 yields 0100.

These operations are the foundation for performing more complex calculations and play a key role in programming and data processing.

The OSINT Framework is a collection of links to various tools and services designed for open source intelligence. On this site, users can find both free and paid resources.

List of tools and services in the OSINT FrameworkScreenshot: OSINT Framework / Skillbox Media

Whois services are Online resources that allow you to find information about domain owners and IP addresses online. Whois results typically contain information about the owner and the company that registered the domain. For example, if you use one of the services to check information about the domain google.com, you will see that it was registered in 1997. The registrar is MarkMonitor, and the registration expires in 2028. It's worth noting that attempts to seize the domain from a company as large as Google would be futile. MarkMonitor, acting as an intermediary for Google in registration matters, renews ownership rights in advance, which makes it impossible to quickly re-register google.com to another owner.

Using the Whois service, you can find out who owns a domain. Screenshot: "Reg.ru" / Skillbox Media

Shodan is a search engine that allows you to discover devices that have access to the internet. This tool scans IP addresses around the world and creates an index of physical objects, such as servers, IoT devices, wireless cameras, routers, and even industrial machines. Simply put, Shodan can be thought of as Google, but for various gadgets.

This tool is most often used by cybersecurity experts to identify vulnerable devices and assess the level of network protection. In addition, Shodan can be very useful in the process of open-source reconnaissance.

When requesting the service, you must specify the server type, its current status, or the IP address of the device, if such information is available. For example, using the request «Server: IP Webcam Server» «200 OK», you can identify available IP cameras.

Open CCTV camera in Seoul. Screenshot: Shodan / Skillbox Media

Maltego is a tool designed to visualize information obtained from open sources. It allows you to analyze collected data and create graphical representations of relationships. For example, using a graph, you can determine which domains are associated with a specific email address.

These connection graphs are built by MaltegoImage: Maltego / Wikimedia Commons

When conducting a manual Data analysis can often be difficult to uncover hidden relationships. Maltego successfully accomplishes this task, highlighting aspects that might otherwise escape a researcher's attention.

SpiderFoot is a tool that automates the process of collecting open-source intelligence (OSINT). This utility can extract information on IP addresses, domain names, email addresses, and usernames, as well as search known leak databases for relevant data. Using SpiderFoot, you can detect vulnerabilities, examine users' digital footprints, and analyze cybersecurity incidents. Upon completion, the tool generates a report displaying all the useful information found in open sources.

SpiderFoot Interface Image: smicallef / GitHub

Have I Been Pwned allows users to find out which data breaches they may be involved in. Simply enter your email address and click the search button. Within seconds, the system will display a list of breaches, along with information about the data that has become publicly available.

List of leaks that included user dataScreenshot: Have I Been Pwned / Skillbox Media

The Have I Been Pwned platform is a tool that can be used not only for collecting Information, but also to ensure personal security. Regularly checking your logins and passwords for leaks can be very useful. For example, if you're registered with SDEK, there's a high probability that your data, including delivery addresses, could be widely available.

TheHarvester is a tool designed to extract data on various companies, widely used by ethical hackers. It can discover employee email addresses, as well as subdomains and IP addresses associated with an organization. This program analyzes information from search engines, open databases, and social networks, accumulating all available data.

TheHarvester utility interface Screenshot: Kali / Skillbox Media

The Boundaries Between Legitimate Intelligence and Criminal Activity in the Context of OSINT

Many people believe that if information is publicly available, it can be used freely and without restrictions. However, the reality is much more complex. Most countries have laws that protect the privacy of citizens. Let's look at the laws that govern open source intelligence (OSINT) in the United States, the European Union, and Russia, as well as the ethical standards that researchers in this field must adhere to.

What's Permitted

  • Research data in publicly available resources, such as Google, social networks, and public registries.
  • Use tools such as Whois, Shodan, Maltego, the OSINT Framework, and others to search and aggregate open information.
  • Examine documents and their metadata, provided that they are publicly available.
  • Use the Have I Been Pwned service to check if your email addresses have been compromised in data breaches.

What's Prohibited

  • Unauthorized access to user accounts and organizational systems.
  • Use social engineering techniques, such as impersonating another person in order to obtain the necessary information. information.
  • Distribute users' personal information without their consent.
  • Access confidential databases without permission.
  • Organize surveillance of the population.

In the United States, it is possible to use open source intelligence (OSINT), but there are certain legal restrictions.

  • The Computer Fraud and Abuse Act (CFAA) prohibits unauthorized access to computer systems and information technology.
  • The Electronic Communications Privacy Act, known as ECPA (Electronic Communications Privacy Act), protects private communications in digital format.
  • The Freedom of Information Act (FOIA) allows citizens to request information from government agencies.

For example, if an OSINT researcher tries to bypass website security measures or download a confidential document, this would be considered a violation of the law.

The European Union has one of the strictest Data protection regulations, known as the General Data Protection Regulation (GDPR), prohibit:

  • Collecting personal information without the knowledge and permission of the person who collects it.
  • Disseminating personal information without the person's consent is unacceptable.
  • Storing information without explicit consent.

Open source researchers must exercise particular caution when conducting work in Europe. Even disclosing a discovered email address could violate the General Data Protection Regulation (GDPR).

Read also:

Artificial Intelligence and Personal Information Protection: How to Ensure Privacy in the Coming World.

Open Sources of Information in Russia (OSINT) operates in a legally uncertain area, as legislation does not always clearly define what is permissible. The process of collecting and storing data is regulated by a number of laws:

  • Federal Law No. 152 "On Personal Data" prohibits the collection and processing of personal information without the consent of its owner.
  • Federal Law No. 149 "On Information, Information Technologies, and Information Protection" establishes rules for access to public data.
  • Federal Law No. 187 "On the Security of Critical Information Infrastructure of the Russian Federation" prohibits unauthorized access to information systems.
  • Certain articles of the Code of Administrative Offenses and the Criminal Code of the Russian Federation may consider OSINT methods as a way to illegally obtain data.

Suppose a researcher finds a publicly available real estate registry and finds out who owns an apartment. If he only familiarizes himself with the information, this is not an offense. However, if he decides to post this data without the owner's permission, he will thereby violate the law regarding the protection of personal data.

In the Russian Federation, doxxing is considered a violation of Article 137 of the Criminal Code, which concerns the protection of privacy.

What to remember

OSINT represents is the ability to identify valuable information where others overlook it. It's not just a Google search skill, but a comprehensive application of various methods that help prevent cyberattacks, detect vulnerabilities, and solve crimes of an international nature.

What's important to understand:

  • Access to information is open to everyone, but the ability to analyze it is key. Finding the necessary data is usually easy, but verifying its reliability and correctly establishing relationships is much more difficult.
  • A number of tools exist that automate the process of extracting data from public sources. These solutions significantly simplify and speed up the work of OSINT researchers.
  • There is a clear distinction between legitimate intelligence and criminal activity. Activities such as account hacking, doxxing, and social engineering constitute violations of the law, not open source research.

You can find even more fascinating information about coding in our Telegram channel. Join us!

Expert in cybersecurity and artificial intelligence

You will be able to develop analytical skills, learn to identify vulnerabilities and ensure the protection of information systems. Even if you have no experience in IT, you can master a sought-after profession. This course program is one of the most comprehensive on the market!

Find out more