Global Technology Monitoring: Making known and unknown technologies visible

Why technology monitoring matters right now

The world is reconfiguring itself. Geopolitical upheaval and a decade-defining foundational technology — artificial intelligence — are rewriting the rules simultaneously. And while the world is still coming to terms with AI, the next wave of transformative technologies is already taking shape on the horizon.

Large parts of the economic system will be forced to reorganise, which poses enormous challenges for everyone in a decision-making role: public institutions that need to shape technology and economic policy, and private companies that need to defend and expand their market position alike.

In this environment, current and reliable market intelligence is absolutely indispensable. Technologies with genuine disruptive potential are of particular relevance here. Mature technologies are already embedded – markets have settled and found equilibrium. But when new technologies emerge, they can open up entirely new markets or disrupt existing ones, with consequences for individual companies, regions, and whole economies. For decision-makers, it is therefore critically important to keep systematic track of novel, transformative, and emerging technologies.

The limits of traditional approaches to technology monitoring

This is precisely where significant methodological challenges come into view – challenges that fall into two core problems.

Problem 1: Known technologies that are invisible in traditional data

Even when you know what technology you're looking for, you run into the problem that emerging technologies simply don't show up in conventional economic data.

• Industry classification codes don't capture novel technology fields. Existing classification systems are rigid, built around established economic structures — not designed to make the new visible. See our study on AI companies in Baden-Württemberg.

• Patent data can partially surface novel technologies, but not all innovations are patented. Patenting is highly relevant in certain sectors, yet it doesn't reflect the full innovation process. In particular, patent data gives little indication of whether technologies have actually been translated into market-ready products.

• Company surveys are another option – but if the relevant actors aren't yet known, those surveys will inevitably be poorly targeted. You're relying on stumbling across a relevant actor by chance, which is especially unlikely when dealing with newly emerging and therefore "rare" technologies.

Problem 2: Finding the unknown

The second problem is more fundamental: what do you do when you don't know what you're looking for at all, and want to work exploratorily?

Emerging technologies evolve constantly. They converge, find one another, and give rise to new fields of application or entirely new research and technology domains – fields that are often very small and hard to identify in their early stages. For exploratory analysis, a temporal dimension is almost essential: it's only through observing change that dynamics become visible, and those dynamics can be the decisive signal pointing toward novel, disruptive technologies.

Traditional datasets are therefore ill-suited both to reliably capturing known emerging technologies and to enabling the exploratory discovery of the unknown.

Global technology monitoring via ISTARI's WebAI approach

This is where ISTARI's WebAI approach comes in – a novel methodology that uses publicly available organisational web data as a source for market and economic intelligence. At its core is the systematic, up-to-the-day analysis of publicly accessible web content (websites, register information, social media, news, annual reports, etc.), processed using artificial intelligence. The advantages can be summarised as follows:

1. Scope and coverage: Analysis of all organisations with a digital footprint, worldwide.

1. Breadth of content: Capturing all publicly communicated technologies, products, services, and more.

1. Timeliness: Reflecting current developments in real time.

1. Dynamics: Repeated analyses make it possible to track change over time.

1. Relational information: Mapping connections, networks, and stakeholder relationships.

This approach, first proposed and substantially developed by ISTARI, is not only finding increasing traction in academic research, but is also feeding into the decision-making processes of institutional actors. Two practical examples follow.

Mapping and international benchmarking of key technologies

The 2026 EFI Report – the annual assessment of Germany's research, innovation, and technological performance, presented to the Federal Chancellor by the Expert Commission on Research and Innovation – includes a section drawing on findings produced using ISTARI's WebAI approach.

The EFI Commission also proposes the WebAI approach as a means of measuring the diffusion of key technologies, specifically in the context of the high-tech agenda Germany. This strategy, adopted by the German federal government in 2025, aims to advance Germany through targeted investment in future technologies. The key technologies it addresses are precisely the kind of emerging, disruptive technologies described above – those with the potential to fundamentally reshape significant parts of the economic system, including artificial intelligence and climate-neutral mobility.

In our companion study to the EFI report, ISTARI examined both AI and climate-neutral mobility. All organisations active or potentially active in these two fields in Germany were identified, providing for the first time a comprehensive picture of how these key technologies have diffused across the German economy. The WebAI approach used is illustrated schematically in the figure below.

The starting point is the ISTARI Global Organization Index (GOI), which contains around 500 million organisations worldwide.

1. Defining the population: Starting from approximately 9 million active organisations in the study's focus countries – Germany, the US, the UK, China, and France *(Step 1 in the visualization)*.

1. Rule-based web crawling: Identifying candidates whose websites display a significant density of relevant keywords *(Steps 2 and 3 in the visualization)*.

1. AI-assisted validation: The identified candidates are then validated by specialised AI agents *(Step 4 in the visualization)*. These agents search the open web – websites, social media, news articles, registers, and further sources – to build a detailed profile for each individual organisation. This step not only filters out irrelevant organisations but also enables detailed analysis and classification of relevant ones: for instance, positioning them along the value chain and identifying relevant products or relationships.

A particular strength of the approach is that the results don't only allow for detailed insights at the level of individual organisations, networks, and regions – they also make it possible to situate national findings within an international context. This enables a peer-group comparison that can show, for example, in which technologies Germany leads internationally and where there is ground to be made up.

Identifying the unknown through exploratory trend analysis

Where the EFI companion study illustrates how the WebAI approach enables targeted searches for known key technologies, the following example demonstrates the second, more fundamental use case: the exploratory identification of the unknown.

As part of a study for the Berlin Senate Department, conducted by ISTARI together with ZEW Mannheim, a workflow was developed to identify trends in the Berlin economy over the past ten years – without any prior specification of which topics to look for. The workflow is illustrated in simplified schematic form below.

The approach breaks down into five steps.

1. Population: The starting point was again the ISTARI Global Organization Index, from which approximately 150,000 active organisations in Berlin were selected.

1. Historical web data: Using an archive of historical website snapshots, snapshots were identified for each of the past ten years for the websites of Berlin-based organisations. This produces not just a cross-section of the current organisational landscape, but a longitudinal dataset that makes change over time visible.

1. Topic analysis: All historical website snapshots were combined into a large text corpus – a comprehensive collection of documents. A topic analysis was then used to identify thematic clusters across all documents. This exploratory method detects thematically similar documents, and therefore organisations that communicate in a thematically comparable way on their websites and consequently share a similar organisational profile – without requiring the relevant topics to be defined in advance.

1. Back-mapping: The identified topics were then mapped back to the individual historical snapshots of each organisation's website. The result: each organisation is assigned relevant topics for each year.

1. Time-series analysis: On the basis of this dataset, a time-series analysis was conducted, aggregating at the annual level to examine how frequently a given topic appeared on organisational websites in any given year. Viewed over time, this surfaces topics whose frequency fluctuates between years – making it possible to identify topics that have grown overall and become increasingly prevalent across Berlin's organisational population, as well as topics that have lost relevance.

This approach thus provides a window into how the relevant themes within an organisational population, whether a region or a particular industry, have shifted over a defined period, without any prior specification of which themes are even worth examining. In the example above, it was possible to identify, among other things, that developers of community-oriented social media platforms and apps have lost significant ground in Berlin over the past decade, while the number of consultancies specialising in AI has grown considerably.

Findings like these have direct implications for economic policy: funding programmes can be revised accordingly and aligned with the real structural shifts taking place in the regional economy – grounded in empirical evidence rather than assumptions. The same holds for companies that need to keep track of their markets and competitors and can't afford to miss a potentially disruptive trend.