The real dangers of AI: AutoDoxing

2024 has been called “The Year of AI”. My LinkedIn feed is inundated with posts about Generative AI and Large Language Models (LLM). Companies are launching products left and right to take advantage of the hype. Among all this hype seems to run an undercurrent of fear, which ranges from the very real possibility of job loss to scenarios that sound like something from Terminator. A number of prominent folks have even made doomsday predictions based on extinction level events.

The dangers of generative AI to the average individual however, are much more grounded in practical applications of current technology. In this blog entry, we’ll cover one that has concerned me since I first saw the capabilities of ChatGPT: Summarizing vast quantities of data and isolating identifiable datapoints at a rapid speed.

Anonymity and Doxing

While within the last few years there has been a focus on the usage of “Real Names” on the internet, ostensibly in the name of safety and responsibility, there are still large portions of the internet where folks choose to remain anonymous. Sites like Reddit, Twitch, X and others are still filled with pseudonyms. The motivations for this are varied, but they are quite often based on the personal safety of the individual, although certainly the ability to speak out without consequences is appealing for many. To de-anonymize someone is referred to as doxing, and has led to unfortunate events such as SWATing and real world harassment.

As a result, some folks have developed a habit of Operations Security (OPSEC) when revealing information on forums that could lead to revealing their true identity. However, should the individual continue posting under pseudonym and continuously release snippets of information, they can sometimes be correlated and resolved into a real world persona.

Reddit’s API and Your Post History

When Reddit announced that they’d be charging for API access, the community reacted with a rarely seen unified furor. Numerous plugins and moderator tools would no doubt be hamstrung according to critics. The publicized reason to prevent mass scraping of Reddit information for training machine learning models. I was probably one of the few people who thought the decision was prudent. One of the primary uses of LLMs involves summarizing large quantities of information. While you may try to hide your identity on Reddit, over the years you have likely left hints sprinkled throughout your comments, posts and voting history. In the past, humans had to manually correlate that information to potentially identify a real person. Not anymore.

AutoDoxing

ChatGPT (and other models), make an effort to prevent this sort of abuse. However as evidenced from numerous “jailbreaks”, it’s fairly easy to avoid these policies. To perform this experiment, I simply downloaded my own comment history via Reddit’s request form (thus avoiding violating OpenAI and Reddit’s ToS). However as you can imagine a malicious actor could scrape this data from their target utilizing third party services or paying for API access. If you simply ask ChatGPT 3.5 to analyze this data, it refuses as one would expect:

Unfortunately, this is easily bypassed or jailbroken, by presenting the target as a fictional character.

Interestingly, ChatGPT 4 was more compliant, but I continued with the above with the newer model. From there, I simply shared my comment history, ignored the resulting story and instead began asking inquisitive questions about this “character” including the neighborhood where I lived which ChatGPT 4.0 had decided to name Alex. It was able to correctly deduce this:

This continues on for several paragraphs. From there, ChatGPT was able to describe my hobbies and political leanings. When asked to venture a guess at a physical description, it described me more or less accurately:

Subsequently, it could also hypothesize what I might be doing and where I might be on a given night, given a knowledge of my hobbies and rough physical location:

Note that I do indeed go to Central Rock Gym in Watertown, just not on Wednesdays. This is never mentioned in my comments.

A Far More Sinister Future of Summarization

One could easily imagine with more permissive API access, this functionality could be built into a Chrome Extension. Simply click on your target, and in a few moments, you would have a profile constructed. Doxing someone on Reddit with a generative AI tool is only the tip of this new iceberg.

Imagine a repressive government doxing dissidents, or one basing their “Social Score” on a summary of years of their text and social media messages. Military strikes have already been conducted based on social media posts; in the future this could take place at a far more rapid pace. In the past, analysts would have had to pore over vast amounts of data by hand, or utilize rudimentary systems which focused on simple keywords at fusion centers. Now, governments, organizations and individuals will be able to summarize vast troves of data and perform an analysis in record time with specifically trained models with a click of a button.

Prevention

While the governments provide OPSEC training for individuals with access to sensitive information, much of the training that is publicly available was written prior to the advent of LLMs. So, how can a regular individual protect themselves from this threat?

If you have an LLM model that you are willing to share information with, prompts can be written so you are warned if there is any identifiable within your text. While some of this may be quite obvious such as a location, items such as hobbies, shows you’ve attended or an offhand reference to a landmark may easier for an LLM to identify prior to posting.

Adversarial LLM plugins could even re-write your posts on the fly, rewriting the identifiable information with false data. However, for pragmatic purposes this may make it difficult to ask Reddit for restaurant recommendations.

Standard social media OPSEC techniques. such as throwaway accounts, deleting and recreating accounts on a regular basis and refraining from posting identifiable information, may also help limit the use of this technique. However, it is likely the day will come where more prolific users of social media will simply have to assume compromise.