This is a multi-part message in MIME format

--_----------=_MCPart_79213603
Content-Type: text/plain; charset="utf-8"; format="fixed" Content-Transfer-Encoding: quoted-printable

** CRYPTO-GRAM
FEBRUARY 15=2C 2025 ------------------------------------------------------------

by Bruce Schneier
Fellow and Lecturer=2C Harvard Kennedy School
schneier@schneier.com
https://www.schneier.com

A free monthly newsletter providing summaries=2C analyses=2C insights=2C a=
nd commentaries on security: computer and otherwise.

For back issues=2C or to subscribe=2C visit Crypto-Gram's web page [https= ://www.schneier.com/crypto-gram/].

Read this issue on the web [https://www.schneier.com/crypto-gram/archives= /2025/0215.html]

These same essays and news items appear in the Schneier on Security [http= s://www.schneier.com/] blog=2C along with a lively and intelligent comment=
section. An RSS feed is available.

** *** ***** ******* *********** *************


** IN THIS ISSUE:
------------------------------------------------------------

1. Phishing False Alarm
2. FBI Deletes PlugX Malware from Thousands of Computers
3. Social Engineering to Disable iMessage Protections
4. Biden Signs New Cybersecurity Order
5. AI Mistakes Are Very Different from Human Mistakes
6. AI Will Write Complex Laws
7. Third Interdisciplinary Workshop on Reimagining Democracy (IWORD 2= 024)
8. New VPN Backdoor
9. CISA Under Trump
10. ExxonMobil Lobbyist Caught Hacking Climate Activists
11. Fake Reddit and WeTransfer Sites Are Pushing Malware
12. Journalists and Civil Society Members Using WhatsApp Targeted by=
Paragon Spyware
13. Deepfakes and the 2024 US Election
14. On Generative AI Security
15. AIs and Robots Should Sound Robotic
16. Screenshot-Reading Malware
17. UK Is Ordering Apple to Break Its Own Encryption
18. Pairwise Authentication of Humans
19. Trusted Execution Environments
20. Delivering Malware Through Abandoned Amazon S3 Buckets
21. DOGE as a National Cyberattack
22. AI and Civil Service Purges
23. Upcoming Speaking Engagements

** *** ***** ******* *********** *************


** PHISHING FALSE ALARM ------------------------------------------------------------

[2025.01.15] [https://www.schneier.com/blog/archives/2025/01/phishing-fa= lse-alarm.html] A very security-conscious company was hit with a (presumed=
) massive state-actor phishing attack with gift cards=2C and everyone rall=
ied to combat it -- until it turned out [https://notalwaysright.com/?p=3D= 359144] it was company management sending the gift cards.

** *** ***** ******* *********** *************


** FBI DELETES PLUGX MALWARE FROM THOUSANDS OF COMPUTERS ------------------------------------------------------------

[2025.01.16] [https://www.schneier.com/blog/archives/2025/01/fbi-deletes= -plugx-malware-from-thousands-of-computers.html] According to a DOJ press=
release [https://www.justice.gov/opa/pr/justice-department-and-fbi-condu= ct-international-operation-delete-malware-used-china-backed]=2C the FBI wa=
s able to delete the Chinese-used PlugX malware from =E2=80=9Capproximatel=
y 4=2C258 U.S.-based computers and networks.=E2=80=9D

Details [https://gizmodo.com/the-fbi-says-it-made-malware-delete-itself-f= rom-americans-computers-2000550046]:

To retrieve information from and send commands to the hacked machines=2C=

the malware connects to a command-and-control server that is operated by=
the hacking group. According to the FBI [https://www.justice.gov/opa/med= ia/1384136/dl]=2C at least 45=2C000 IP addresses in the US had back-and-fo= rths with the command-and-control server since September 2023.

It was that very server that allowed the FBI to finally kill this pesky=

bit of malicious software. First=2C they tapped the know-how of French in= telligence agencies=2C which had recently discovered a technique [https:/= /www.bleepingcomputer.com/news/security/french-police-push-plugx-malware-s= elf-destruct-payload-to-clean-pcs/] for getting PlugX to self-destruct. Th= en=2C the FBI gained access to the hackers=E2=80=99 command-and-control se= rver and used it to request all the IP addresses of machines that were act= ively infected by PlugX. Then it sent a command via the server that causes=
PlugX to delete itself from its victims=E2=80=99 computers.

** *** ***** ******* *********** *************


** SOCIAL ENGINEERING TO DISABLE IMESSAGE PROTECTIONS ------------------------------------------------------------

[2025.01.17] [https://www.schneier.com/blog/archives/2025/01/social-engi= neering-to-disable-imessage-protections.html] I am always interested in ne=
w phishing tricks=2C and watching them spread across the ecosystem.

A few days ago I started getting phishing SMS messages with a new twist. T=
hey were standard messages about delayed packages or somesuch=2C with the=
goal of getting me to click on a link and entering some personal informat=
ion into a website. But because they came from unknown phone numbers=2C th=
e links did not work. So -- this is the new bit -- the messages said somet= hing like: =E2=80=9CPlease reply Y=2C then exit the text message=2C reopen=
the text message activation link=2C or copy the link to Safari browser to=
open it.=E2=80=9D

I saw it once=2C and now I am seeing it again and again. Everyone has now=
adopted this new trick.

One article claims that this trick has been popular since last summer [ht= tps://www.bleepingcomputer.com/news/security/phishing-texts-trick-apple-im= essage-users-into-disabling-protection/]. I don=E2=80=99t know; I would ha=
ve expected to have seen it before last weekend.

** *** ***** ******* *********** *************


** BIDEN SIGNS NEW CYBERSECURITY ORDER ------------------------------------------------------------

[2025.01.20] [https://www.schneier.com/blog/archives/2025/01/biden-signs= -new-cybersecurity-order.html] President Biden has signed a new cybersecur=
ity order [https://bidenwhitehouse.archives.gov/briefing-room/presidentia= l-actions/2025/01/16/executive-order-on-strengthening-and-promoting-innova= tion-in-the-nations-cybersecurity/]. It has a bunch of provisions=2C most=
notably using the US government=E2=80=99s procurement power to improve cy= bersecurity practices industry-wide.

Some details [https://www.wired.com/story/biden-executive-order-cybersecu= rity-ai-and-more/]:

The core of the executive order is an array of mandates for protecting g=

overnment networks based on lessons learned from recent major incidents --=
namely=2C the security failures of federal contractors.

The order requires software vendors to submit proof that they follow sec=

ure development practices=2C building on a mandate that debuted [https://= www.whitehouse.gov/wp-content/uploads/2022/09/M-22-18.pdf] in 2022 in resp= onse to Biden=E2=80=99s first cyber executive order [https://www.whitehou= se.gov/briefing-room/presidential-actions/2021/05/12/executive-order-on-im= proving-the-nations-cybersecurity/]. The Cybersecurity and Infrastructure=
Security Agency would be tasked with double-checking these security attes= tations and working with vendors to fix any problems. To put some teeth be= hind the requirement=2C the White House=E2=80=99s Office of the National C= yber Director is =E2=80=9Cencouraged to refer attestations that fail valid= ation to the Attorney General=E2=80=9D for potential investigation and pro= secution.

The order gives the Department of Commerce eight months to assess the mo=

st commonly used cyber practices in the business community and issue guida=
nce based on them. Shortly thereafter=2C those practices would become mand= atory for companies seeking to do business with the government. The direct=
ive also kicks off updates to the National Institute of Standards and Tech= nology=E2=80=99s secure software development guidance. [https://csrc.nist= =2Egov/projects/ssdf]

More [https://therecord.media/biden-cybersecurity-executive-order] inform= ation [https://www.csoonline.com/article/3802476/biden-white-house-to-go-= all-out-in-final-sweeping-cybersecurity-order.html].

** *** ***** ******* *********** *************


** AI MISTAKES ARE VERY DIFFERENT FROM HUMAN MISTAKES ------------------------------------------------------------

[2025.01.21] [https://www.schneier.com/blog/archives/2025/01/ai-mistakes= -are-very-different-from-human-mistakes.html] Humans make mistakes all the=
time. All of us do=2C every day=2C in tasks both new and routine. Some of=
our mistakes are minor and some are catastrophic. Mistakes can break trus=
t with our friends=2C lose the confidence of our bosses=2C and sometimes b=
e the difference between life and death.

Over the millennia=2C we have created security systems to deal with the so=
rts of mistakes humans commonly make. These days=2C casinos rotate their d= ealers regularly=2C because they make mistakes if they do the same task fo=
r too long. Hospital personnel write on limbs before surgery so that docto=
rs operate on the correct body part=2C and they count surgical instruments=
to make sure none were left inside the body. From copyediting to double-e= ntry bookkeeping to appellate courts=2C we humans have gotten really good=
at correcting human mistakes.

Humanity is now rapidly integrating a wholly different kind of mistake-mak=
er into society: AI. Technologies like large language models [https://spe= ctrum.ieee.org/tag/llms] (LLMs) can perform many cognitive tasks tradition= ally fulfilled by humans=2C but they make plenty of mistakes. It seems rid= iculous [https://www.buzzfeed.com/carleysuthers/weird-and-wrong-ai-respon=
ses] when chatbots tell you to eat rocks or add glue to pizza. But it=E2= =80=99s not the frequency or severity of AI systems=E2=80=99 mistakes that=
differentiates them from human mistakes. It=E2=80=99s their weirdness. AI=
systems do not make mistakes in the same ways that humans do.

Much of the friction -- and risk -- associated with our use of AI arise fr=
om that difference. We need to invent new security [https://spectrum.ieee= =2Eorg/tag/security] systems that adapt to these differences and prevent har=
m from AI mistakes.

* HUMAN MISTAKES VS AI MISTAKES

Life experience makes it fairly easy for each of us to guess when and wher=
e humans will make mistakes. Human errors tend to come at the edges of som= eone=E2=80=99s knowledge: Most of us would make mistakes solving calculus=
problems. We expect human mistakes to be clustered: A single calculus mis= take is likely to be accompanied by others. We expect mistakes to wax and=
wane=2C predictably depending on factors such as fatigue and distraction.=
And mistakes are often accompanied by ignorance: Someone who makes calcul=
us mistakes is also likely to respond =E2=80=9CI don=E2=80=99t know=E2=80=
=9D to calculus-related questions.

To the extent that AI systems make these human-like mistakes=2C we can bri=
ng all of our mistake-correcting systems to bear on their output. But the=
current crop of AI models -- particularly LLMs -- make mistakes different=
ly.

AI errors come at seemingly random times=2C without any clustering around=
particular topics. LLM mistakes tend to be more evenly distributed throug=
h the knowledge space. A model might be equally likely to make a mistake o=
n a calculus question as it is to propose that cabbages [https://arxiv.or= g/html/2405.19616v1] eat goats.

And AI mistakes aren=E2=80=99t accompanied by ignorance. A LLM will be jus=
t as confident [https://spectrum.ieee.org/chatgpt-reliability] when sayin=
g something completely wrong -- and obviously so=2C to a human -- as it wi=
ll be when saying something true. The seemingly random inconsistency [htt= ps://arxiv.org/pdf/2305.14279] of LLMs makes it hard to trust their reason=
ing in complex=2C multi-step problems. If you want to use an AI model to h=
elp with a business problem=2C it=E2=80=99s not enough to see that it unde= rstands what factors make a product profitable; you need to be sure it won= =E2=80=99t forget what money is.

* HOW TO DEAL WITH AI MISTAKES

This situation indicates two possible areas of research. The first is to e= ngineer LLMs that make more human-like mistakes. The second is to build ne=
w mistake-correcting systems that deal with the specific sorts of mistakes=
that LLMs tend to make.

We already have some tools to lead LLMs to act in more human-like ways. Ma=
ny of these arise from the field of =E2=80=9Calignment [https://arxiv.org= /abs/2406.18346]=E2=80=9D research=2C which aims to make models act in acc= ordance [https://spectrum.ieee.org/the-alignment-problem-openai] with the=
goals and motivations of their human developers. One example is the techn= ique that was arguably [https://venturebeat.com/ai/how-reinforcement-lear= ning-with-human-feedback-is-unlocking-the-power-of-generative-ai/] respons= ible for the breakthrough success of ChatGPT [https://spectrum.ieee.org/t= ag/chatgpt]: reinforcement learning with human feedback [https://arxiv.or= g/abs/2203.02155]. In this method=2C an AI model is (figuratively) rewarde=
d for producing responses that get a thumbs-up from human evaluators. Simi=
lar approaches could be used to induce AI systems to make more human-like=
mistakes=2C particularly by penalizing them more for mistakes that are le=
ss intelligible.

When it comes to catching AI mistakes=2C some of the systems that we use t=
o prevent human mistakes will help. To an extent=2C forcing LLMs to double= -check [https://arxiv.org/pdf/2308.00436] their own work can help prevent=
errors. But LLMs can also confabulate [https://arxiv.org/pdf/2406.02061]=
seemingly plausible=2C but truly ridiculous=2C explanations for their fli= ghts from reason.

Other mistake mitigation systems for AI are unlike anything we use for hum= ans. Because machines can=E2=80=99t get fatigued or frustrated in the way=
that humans do=2C it can help to ask an LLM the same question repeatedly=
in slightly different ways and then synthesize [https://arxiv.org/abs/22= 10.02441] its multiple responses. Humans won=E2=80=99t put up with that ki=
nd of annoying repetition=2C but machines will.

* UNDERSTANDING SIMILARITIES AND DIFFERENCES

Researchers are still struggling to understand where LLM mistakes diverge=
from human ones. Some of the weirdness of AI is actually more human-like=
than it first appears. Small changes to a query to an LLM can result in w= ildly different responses=2C a problem known as prompt sensitivity [https= ://arxiv.org/pdf/2311.07230]. But=2C as any survey researcher can tell you=
=2C humans behave this way=2C too. The phrasing of a question in an opinio=
n poll can have drastic impacts [https://psycnet.apa.org/record/1992-9732= 9-001] on the answers.

LLMs also seem to have a bias towards repeating [http://proceedings.mlr.p= ress/v139/zhao21c/zhao21c.pdf] the words that were most common in their tr= aining data; for example=2C guessing familiar place names like =E2=80=9CAm= erica=E2=80=9D even when asked about more exotic locations. Perhaps this i=
s an example of the human =E2=80=9Cavailability heuristic [https://arxiv.= org/pdf/2305.04400]=E2=80=9D manifesting in LLMs=2C with machines spitting=
out the first thing that comes to mind rather than reasoning through the=
question. And like humans=2C perhaps=2C some LLMs seem to get distracted=
[https://arxiv.org/html/2404.08865v1] in the middle of long documents; t= hey=E2=80=99re better able to remember facts from the beginning and end. T= here is already progress on improving this error mode=2C as researchers ha=
ve found that LLMs trained on more examples [https://www.anthropic.com/ne= ws/claude-2-1-prompting] of retrieving information from long texts seem to=
do better at retrieving information uniformly.

In some cases=2C what=E2=80=99s bizarre about LLMs is that they act more l=
ike humans than we think they should. For example=2C some researchers have=
tested the hypothesis [https://minimaxir.com/2024/02/chatgpt-tips-analys=
is/] that LLMs perform better when offered a cash reward or threatened wit=
h death. It also turns out that some of the best ways to =E2=80=9Cjailbrea=
k [https://www.usenix.org/system/files/sec24fall-prepub-1500-yu-zhiyuan.p= df]=E2=80=9D LLMs (getting them to disobey their creators=E2=80=99 explici=
t instructions) look a lot like the kinds of social engineering tricks tha=
t humans use on each other: for example=2C pretending to be someone else o=
r saying that the request is just a joke. But other effective jailbreaking=
techniques are things no human would ever fall for. One group found [htt= ps://arxiv.org/abs/2402.11753] that if they used ASCII art [https://en.wi= kipedia.org/wiki/ASCII_art] (constructions of symbols that look like words=
or pictures) to pose dangerous questions=2C like how to build a bomb=2C t=
he LLM would answer them willingly.

Humans may occasionally make seemingly random=2C incomprehensible=2C and i= nconsistent mistakes=2C but such occurrences are rare and often indicative=
of more serious problems. We also tend not to put people exhibiting these=
behaviors in decision-making positions. Likewise=2C we should confine AI=
decision-making systems to applications that suit their actual abilities=
-- while keeping the potential ramifications of their mistakes firmly in=
mind.

_This essay was written with Nathan E. Sanders=2C and originally appeared=
in IEEE Spectrum [https://spectrum.ieee.org/ai-mistakes-schneier]._

EDITED TO ADD (1/24): Slashdot thread [https://slashdot.org/story/25/01/2= 3/1645242/ai-mistakes-are-very-different-from-human-mistakes].

** *** ***** ******* *********** *************


** AI WILL WRITE COMPLEX LAWS ------------------------------------------------------------

[2025.01.22] [https://www.schneier.com/blog/archives/2025/01/ai-will-wri= te-complex-laws.html] Artificial intelligence (AI) is writing law today. T=
his has required no changes in legislative procedure or the rules of legis= lative bodies -- all it takes is one legislator=2C or legislative assistan= t=2C to use generative AI in the process of drafting a bill.

In fact=2C the use of AI by legislators is only likely to become more prev= alent. There are currently projects in the US House=2C US Senate=2C and le= gislatures around the world [https://www.popvox.org/blog/assessing-us-con= gressional-ai-adoption] to trial the use of AI in various ways: searching=
databases=2C drafting text=2C summarizing meetings=2C performing policy r= esearch and analysis=2C and more. A Brazilian municipality passed the firs=
t known AI-written law [https://apnews.com/article/brazil-artificial-inte= lligence-porto-alegre-5afd1240afe7b6ac202bb0bbc45e08d4] in 2023.

That=E2=80=99s not surprising; AI is being used more everywhere. What is c= oming into focus is how policymakers will use AI and=2C critically=2C how=
this use will change the balance of power between the legislative and exe= cutive branches of government. Soon=2C US legislators may turn to AI to he=
lp them keep pace with the increasing complexity of their lawmaking -- and=
this will suppress the power and discretion of the executive branch to ma=
ke policy.

* DEMAND FOR INCREASINGLY COMPLEX LEGISLATION

Legislators are writing increasingly long=2C intricate=2C and complicated=
laws that human legislative drafters have trouble producing. Already in t=
he US=2C the multibillion-dollar lobbying industry is subsidizing lawmaker=
s [https://www.cambridge.org/core/journals/american-political-science-rev= iew/article/abs/lobbying-as-legislative-subsidy/AE4B5D8AB9C2487BB78C2A51BB= 53E03F] in writing baroque laws: suggesting paragraphs to add to bills=2C=
specifying benefits for some=2C carving out exceptions for others. Indeed=
=2C the lobbying industry [https://www.oecd.org/en/publications/lobbying-= in-the-21st-century_c6d8eff8-en.html] is growing in complexity and influen=
ce worldwide.

Several years ago=2C researchers studied bills [http://dx.doi.org/10.1177= /1532673X18776628] introduced into state legislatures throughout the US=2C=
looking at which bills were wholly original texts and which borrowed text=
from other states or from lobbyist-written model legislation. Their concl= usion was not very surprising. Those who borrowed the most text were in le= gislatures that were less resourced. This makes sense: If you=E2=80=99re a=
part-time legislator=2C perhaps unpaid and without a lot of staff=2C you=
need to rely on more external support to draft legislation. When the scop=
e of policymaking outstrips the resources of legislators=2C they look for=
help. Today=2C that often means lobbyists=2C who provide expertise=2C res= earch services=2C and drafting labor to legislators at the local=2C state=
=2C and federal levels at no charge. Of course=2C they are not unbiased: T=
hey seek to exert influence on behalf of their clients.

Ano [https://doi.org/10.1086/714933]ther study [https://doi.org/10.1086/= 714933]=2C at the US federal level=2C measured the complexity of policies=
proposed in legislation and tried to determine the factors that led to su=
ch growing complexity. While there are numerous ways [https://core.ac.uk/= download/pdf/217430499.pdf] to measure legal complexity=2C these authors f= ocused on the specificity of institutional design: How exacting is Congres=
s in laying out the relational network of branches=2C agencies=2C and offi= cials that will share power to implement the policy?

In looking at bills enacted between 1993 and 2014=2C the researchers found=
two things. First=2C they concluded that ideological polarization drives=
complexity. The suggestion is that if a legislator is on the extreme end=
of the ideological spectrum=2C they=E2=80=99re more likely to introduce a=
complex law that constrains the discretion of=2C as the authors put it=2C=
=E2=80=9Centrenched bureaucratic interests.=E2=80=9D And second=2C they f= ound that divided government drives complexity to a large degree: Signific=
ant legislation passed under divided government was found to be 65 percent=
more complex than similar legislation passed under unified government. Th=
eir conclusion is that=2C if a legislator=E2=80=99s party controls Congres= s=2C and the opposing party controls the White House=2C the legislator wil=
l want to give the executive as little wiggle room as possible. When legis= lators=E2=80=99 preferences disagree with the executive=E2=80=99s=2C the l= egislature is incentivized to write laws that specify all the details. Thi=
s gives the agency designated to implement the law as little discretion as=
possible.

Because polarization and divided government are increasingly entrenched in=
the US=2C the demand for complex legislation at the federal level is like=
ly to grow. Today=2C we have both the greatest ideological polarization [= https://www.pewresearch.org/short-reads/2022/03/10/the-polarization-in-tod= ays-congress-has-roots-that-go-back-decades] in Congress in living memory=
and an increasingly divided government at the federal level. Between 1900=
and 1970 (57th through 90th Congresses)=2C we had 27 instances of unified=
government and only seven divided; nearly a four-to-one ratio. Since then=
=2C the trend is roughly the opposite. As of the start of the next Congres= s=2C we will have had 20 divided governments and only eight unified (nearl=
y a three-to-one ratio). And while the incoming Trump administration will=
see a unified government=2C the extremely closely divided House may often=
make this Congress look and feel like a divided one (see the recent gover= nment shutdown crisis [https://www.washingtonpost.com/politics/2024/12/21= /trump-musk-shutdown-demands-defeat/] as an exemplar) and makes truly divi=
ded government a strong possibility in 2027.

Another related factor driving the complexity of legislation is the need t=
o do it all at once. The lobbyist feeding frenzy [https://onlinelibrary.w= iley.com/doi/full/10.1111/lsq.12266] -- spurring major bills like the Affo= rdable Care Act to be thousands of pages in length -- is driven in part by=
gridlock in Congress. Congressional productivity has dropped so low [htt= ps://abcnews.go.com/Politics/118th-congress-track-become-productive-us-his= tory/story?id=3D106254012] that bills on any given policy issue seem like=
a once-in-a-generation opportunity for legislators -- and lobbyists -- to=
set policy.

These dynamics also impact the states. States often have divided governmen= ts=2C albeit less often than they used to [https://www.multistate.us/insi= der/2024/2/13/state-trifectas-hit-a-new-record-in-2024-only-10-states-have= -divided-government]=2C and their demand for drafting assistance is arguab=
ly higher due to their significantly smaller staffs. And since the product= ivity of Congress has cratered [https://www.reuters.com/graphics/USA-CONG= RESS/PRODUCTIVITY/egpbabmkwvq/] in recent years=2C significantly more poli= cymaking is happening at the state level.

But there=E2=80=99s another reason=2C particular to the US federal governm= ent=2C that will likely force congressional legislation to be more complex=
even during unified government. In June 2024=2C the US Supreme Court over= turned the [https://www.scotusblog.com/2024/06/supreme-court-strikes-dow= n-chevron-curtailing-power-of-federal-agencies]_Chevron_ [https://www.sco= tusblog.com/2024/06/supreme-court-strikes-down-chevron-curtailing-power-of= -federal-agencies] doctrine [https://www.scotusblog.com/2024/06/supreme-c= ourt-strikes-down-chevron-curtailing-power-of-federal-agencies]=2C which g=
ave executive agencies broad power to specify and implement legislation. S= uddenly=2C there is a mandate from the Supreme Court [https://www.klgates= =2Ecom/What-Overturning-Chevron-Means-for-the-Way-Congress-Does-Its-Business= -7-18-2024] for more specific legislation. Issues that have historically b=
een left implicitly to the executive branch are now required to be either=
explicitly delegated to agencies or specified directly in statute. Either=
way=2C the Court=E2=80=99s ruling implied that law should become more com= plex and that Congress should increase its policymaking capacity [https:/= /bipartisanpolicy.org/blog/building-congress-for-post-chevron-world/].

This affects the balance of power between the executive and legislative br= anches of government. When the legislature delegates less to the executive=
branch=2C it increases its own power. Every decision made explicitly in s= tatute is a decision the executive makes not on its own but=2C rather=2C a= ccording to the directive of the legislature. In the US system of separati=
on of powers [https://core.ac.uk/download/pdf/80562076.pdf]=2C administra=
tive law is a tool for balancing power among the legislative=2C executive=
=2C and judicial branches. The legislature gets to decide when to delegate=
and when not to=2C and it can respond to judicial review to adjust its de= legation of control as needed. The elimination of _Chevron_ will induce th=
e legislature to exert its control over delegation more robustly.

At the same time=2C there are powerful political incentives for Congress t=
o be vague [https://goodauthority.org/news/abolishing-chevron-could-under= mine-congress-scotus-loper] and to rely on someone else=2C like agency bur= eaucrats=2C to make hard decisions. That empowers third parties -- the cor= porations=2C or lobbyists -- that have been gifted by the overturning of _= Chevron_ a new tool in arguing against administrative regulations not spec= ifically backed up by law. A continuing stream of Supreme Court decisions=
[https://www.nytimes.com/2024/06/28/us/politics/supreme-court-regulatory= -agencies.html] handing victories to unpopular industries could be another=
driver of complex law=2C adding political pressure to pass legislative fi= xes.

* AI CAN SUPPLY COMPLEX LEGISLATION

Congress may or may not be up to the challenge of putting more policy deta=
ils into law=2C but the external forces outlined above -- lobbyists=2C the=
judiciary=2C and an increasingly divided and polarized government -- are=
pushing them to do so. When Congress does take on the task of writing com= plex legislation=2C it=E2=80=99s quite likely it will turn to AI for help.

Two particular AI capabilities enable Congress to write laws different fro=
m laws humans tend to write. One=2C AI models have an enormous _scope_ of=
expertise=2C whereas people have only a handful of specializations. Large=
language models (LLMs) like the one powering ChatGPT can generate legisla= tive text on funding specialty crop harvesting mechanization equally as we=
ll as material on energy efficiency standards for street lighting. This en= ables a legislator to address more topics simultaneously. Two=2C AI models=
have the _sophistication_ to work with a higher degree of complexity than=
people can. Modern LLM systems can instantaneously perform several simult= aneous multistep reasoning tasks [https://arxiv.org/pdf/2411.05000] using=
information from thousands of pages of documents. This enables a legislat=
or to fill in more baroque detail on any given topic.

That=E2=80=99s not to say that handing over legislative drafting to machin=
es is easily done. Modernizing any institutional process is extremely hard=
[https://www.recodingamerica.us]=2C even when the technology is readily=
available and performant. And modern AI still has a ways to go [https://= proceedings.neurips.cc/paper_files/paper/2023/file/89e44582fd28ddfea1ea4dc= b0ebbf4b0-Paper-Datasets_and_Benchmarks.pdf] to achieve mastery of complex=
legal and policy issues. But the basic tools are there.

AI can be used in each step of lawmaking=2C and this will bring various be= nefits to policymakers. It could let them work on more policies -- more bi=
lls -- at the same time=2C add more detail and specificity to each bill=2C=
or interpret and incorporate more feedback from constituents [https://st= atic.ie.edu/CGC/AI4D%20Paper%203%20Applications%20of%20Artificial%20Intell= igence%20Tools%20to%20Engance%20Legislative%20Engagement.pdf] and outside=
groups. The addition of a single AI tool to a legislative office may have=
an impact similar to adding several people to their staff=2C but with far=
lower cost.

Speed sometimes matters when writing law. When there is a change of govern=
ing party=2C there is often a rush to change as much policy as possible to=
match the platform of the new regime. AI could help legislators do that k=
ind of wholesale revision. The result could be policy that is more respons=
ive to voters -- or more political instability. Already in 2024=2C the US=
House=E2=80=99s Office of the Clerk has begun using AI [https://cha.hous= e.gov/_cache/files/7/5/75fda5da-b1e4-4990-a543-6ca495f983ef/CD88E4BBA4CE5A= BDD53A31C31931168B.cha-modernization-ai-flash-report-07-30-24-51-.pdf] to=
speed up the process of producing cost estimates for bills and understand=
ing how new legislation relates to existing code. Ohio has used an AI tool=
[https://search-prod.lis.state.oh.us/api/v2/general_assembly_135/committ= ees/cmte_s_govt_1/meetings/cmte_s_govt_1_2024-05-22-1030_1242/submissions/= olr_testimony_csi.pdf] to do wholesale revision of state administrative la=
w since 2020.

AI can also make laws clearer and more consistent. With their superhuman a= ttention spans=2C AI tools are good at enforcing syntactic and grammatical=
rules. They will be effective at drafting text in precise and proper legi= slative language=2C or offering detailed feedback to human drafters. Borro= wing ideas from software development=2C where coders use tools to identify=
common instances of bad programming practices=2C an AI reviewer can highl= ight bad law-writing practices [https://link.springer.com/article/10.1007= /s10506-022-09315-w]. For example=2C it can detect when significant phrasi=
ng is inconsistent across a long bill. If a bill about insurance repeatedl=
y lists a variety of disaster categories=2C but leaves one out one time=2C=
AI can catch that.

Perhaps this seems like minutiae=2C but a small ambiguity or mistake in la=
w can have massive consequences. In 2015=2C the Affordable Care Act came c= lose to being struck down because of a typo in four words [https://www.ny= times.com/2015/05/26/us/politics/contested-words-in-affordable-care-act-ma= y-have-been-left-by-mistake.html]=2C imperiling health care services exten=
ded to more than 7 million Americans.

There=E2=80=99s more that AI can do in the legislative process. AI can sum= marize bills and answer questions about their provisions. It can highlight=
aspects of a bill that align with=2C or are contrary to=2C different poli= tical points of view. We can even imagine a future in which AI can be used=
to simulate a new law and determine whether or not it would be effective=
=2C or what the side effects would be. This means that beyond writing them=
=2C AI could help lawmakers _understand_ laws. Congress is notorious for p= roducing bills hundreds of pages long=2C and many other countries sometime=
s have similarly massive omnibus bills that address many issues at once. I= t=E2=80=99s impossible for any one person to understand how each of these=
bills=E2=80=99 provisions would work. Many legislatures employ human anal= ysis in budget or fiscal offices that analyze these bills and offer report=
s. AI could do this kind of work at greater speed and scale=2C so legislat=
ors could easily query an AI tool about how a particular bill would affect=
their district or areas of concern.

This is a use case that the House [https://cha.house.gov/modernization]s= ubcommittee [https://cha.house.gov/modernization] on modernization has ur=
ged the Library of Congress [https://cha.house.gov/_cache/files/0/8/08476= 380-95c3-4989-ad4c-1e2a454b0007/9668ADB6A0D503B944E26EDB81EDC585.cha-moder= nization-ai-flash-report-10-25-24.pdf] to take action on. Numerous softwar=
e vendors are already marketing AI legislative [https://www.law.com/legal= technews/2024/11/22/legaltech-rundown-lexisnexis-releases-lexis-ai-mobile-= app-hotshot-launches-new-ma-training-simulation-and-more] analysis tools.=
These tools can potentially find loopholes or=2C like the human lobbyists=
of today [https://www.technologyreview.com/2023/03/14/1069717/how-ai-cou= ld-write-our-laws/]=2C craft them to benefit particular private interests.

These capabilities will be attractive to legislators who are looking to ex= pand their power and capabilities but don=E2=80=99t necessarily have more=
funding to hire human staff. We should understand the idea of AI-augmente=
d lawmaking contextualized within the longer history of legislative techno= logies. To serve society at modern scales=2C we=E2=80=99ve had to come a l=
ong way from the Athenian ideals of direct democracy and sortition. Democr=
acy no longer involves just one person and one vote to decide a policy. It=
involves hundreds of thousands of constituents electing one representativ= e=2C who is augmented by a staff as well as subsidized by lobbyists=2C and=
who implements policy through a vast administrative state coordinated by=
digital technologies. Using AI to help those representatives specify and=
refine their policy ideas is part of a long history of transformation.

Whether all this AI augmentation is good for all of us subject to the laws=
they make is less clear. There are real risks to AI-written law=2C but th=
ose risks are not dramatically different from what we endure today. AI-wri= tten law trying to optimize for certain policy outcomes may get it wrong (= just as many human-written laws are misguided). AI-written law may be mani= pulated to benefit one constituency over others=2C by the tech companies t=
hat develop the AI=2C or by the legislators who apply it=2C just as human=
lobbyists steer policy to benefit their clients.

Regardless of what anyone thinks of any of this=2C regardless of whether i=
t will be a net positive or a net negative=2C AI-made legislation is comin=
g -- the growing complexity of policy demands it. It doesn=E2=80=99t requi=
re any changes in legislative procedures or agreement from any rules commi= ttee. All it takes is for one legislative assistant=2C or lobbyist=2C to f=
ire up a chatbot and ask it to create a draft. When legislators voted on t=
hat Brazilian bill in 2023=2C they didn=E2=80=99t know it was AI-written [= https://apnews.com/article/brazil-artificial-intelligence-porto-alegre-5a= fd1240afe7b6ac202bb0bbc45e08d4]; the use of ChatGPT was undisclosed. And e=
ven if they had known=2C it=E2=80=99s not clear it would have made a diffe= rence. In the future=2C as in the past=2C we won=E2=80=99t always know whi=
ch laws will have good impacts and which will have bad effects=2C regardle=
ss of the words on the page=2C or who (or what) wrote them.

_This essay was written with Nathan E. Sanders=2C and originally appeared=
in Lawfare [https://www.lawfaremedia.org/article/ai-will-write-complex-l= aws]._

** *** ***** ******* *********** *************


** THIRD INTERDISCIPLINARY WORKSHOP ON REIMAGINING DEMOCRACY (IWORD 2024) ------------------------------------------------------------

[2025.01.23] [https://www.schneier.com/blog/archives/2025/01/third-inter= disciplinary-workshop-on-reimagining-democracy-iword-2024.html] Last month=
=2C Henry Farrell and I convened the Third Interdisciplinary Workshop on R= eimagining Democracy (IWORD 2024 [https://www.schneier.com/iword/2024/])=
at Johns Hopkins University=E2=80=99s Bloomberg Center in Washington DC.=
This is a small=2C invitational workshop on the future of democracy. As w=
ith the previous [https://www.schneier.com/iword/2022/] two [https://www= =2Eschneier.com/iword/2023/] workshops=2C the goal was to bring together a d= iverse set of political scientists=2C law professors=2C philosophers=2C AI=
researchers and other industry practitioners=2C political activists=2C an=
d creative types (including science fiction writers) to discuss how democr=
acy might be reimagined in the current century.

The goal of the workshop is to think very broadly. Modern democracy was in= vented in the mid-eighteenth century=2C using mid-eighteenth-century techn= ology. If democracy were to be invented today=2C it would look very differ= ent. Elections would look different. The balance between representation an=
d direct democracy would look different. Adjudication and enforcement woul=
d look different. Everything would look different=2C because our conceptio=
ns of fairness=2C justice=2C equality=2C and rights are different=2C and w=
e have much more powerful technology to bring to bear on the problems. Als= o=2C we could start from scratch without having to worry about evolving ou=
r current democracy into this imagined future system.

We can=E2=80=99t do that=2C of course=2C but it=E2=80=99s still still valu= able to speculate. Of course we need to figure out how to reform our curre=
nt systems=2C but we shouldn=E2=80=99t limit our thinking to incremental s= teps. We also need to think about discontinuous changes as well. I wrote a= bout the philosophy more in this essay [https://theconversation.com/re-im= agining-democracy-for-the-21st-century-possibly-without-the-trappings-of-t= he-18th-century-210586] about IWORD 2022.

IWORD 2024 was easily the most intellectually stimulating two days of my y= ear. It=E2=80=99s also intellectually exhausting; the speed and intensity=
of ideas is almost too much. I wrote about the format in my blog post [h= ttps://www.schneier.com/blog/archives/2024/01/second-interdisciplinary-wor= kshop-on-reimagining-democracy.html] on IWORD 2023.

Summaries of all the IWORD 2024 talks are in the first set of comments bel=
ow. And here are links to the previous IWORDs:

* IWORD 2022: home page [https://www.schneier.com/iword/2022/]=2C es=
say [https://theconversation.com/re-imagining-democracy-for-the-21st-cent= ury-possibly-without-the-trappings-of-the-18th-century-210586]=2C and talk=
summaries [https://www.schneier.com/blog/archives/2022/12/reimagining-de= mocracy.html]
* IWORD 2023: home page [https://www.schneier.com/iword/2023/] and t=
alk summaries [https://www.schneier.com/blog/archives/2024/01/second-inte= rdisciplinary-workshop-on-reimagining-democracy.html].

IWORD 2025 will be held either in New York or New Haven; still to be deter= mined.

** *** ***** ******* *********** *************


** NEW VPN BACKDOOR ------------------------------------------------------------

[2025.01.27] [https://www.schneier.com/blog/archives/2025/01/new-vpn-bac= kdoor.html] A newly discovered VPN backdoor [https://arstechnica.com/secu= rity/2025/01/backdoor-infecting-vpns-used-magic-packets-for-stealth-and-se= curity/] uses some interesting tactics to avoid detection:

When threat actors use backdoor malware to gain access to a network=2C t=

hey want to make sure all their hard work can=E2=80=99t be leveraged by co= mpeting groups or detected by defenders. One countermeasure is to equip th=
e backdoor with a passive agent that remains dormant until it receives wha= t=E2=80=99s known in the business as a =E2=80=9Cmagic packet.=E2=80=9D On=
Thursday=2C researchers revealed that a never-before-seen backdoor that q= uietly took hold of dozens of enterprise VPNs running Juniper Network=E2= =80=99s Junos OS has been doing just that.

J-Magic=2C the tracking name for the backdoor=2C goes one step further t=

o prevent unauthorized access. After receiving a magic packet hidden in th=
e normal flow of TCP traffic=2C it relays a challenge to the device that s=
ent it. The challenge comes in the form of a string of text that=E2=80=99s=
encrypted using the public portion of an RSA key. The initiating party mu=
st then respond with the corresponding plaintext=2C proving it has access=
to the secret key.

The lightweight backdoor is also notable because it resided only in memo=

ry=2C a trait that makes detection harder for defenders. The combination p= rompted researchers at Lumin Technology=E2=80=99s Black Lotus Lab to sit u=
p and take notice.

[...]

The researchers found J-Magic on VirusTotal [https://www.virustotal.com=

/gui/home/upload] and determined that it had run inside the networks of 36=
organizations. They still don=E2=80=99t know how the backdoor got install=
ed.

Slashdot thread [https://tech.slashdot.org/story/25/01/24/0039249/backdoo= r-infecting-vpns-used-magic-packets-for-stealth-and-security].

EDITED TO ADD (2/1): Another article [https://www.theregister.com/2025/01= /25/mysterious_backdoor_juniper_routers/].

** *** ***** ******* *********** *************


** CISA UNDER TRUMP ------------------------------------------------------------

[2025.01.28] [https://www.schneier.com/blog/archives/2025/01/cisa-under-= trump.html] Jen Easterly is out as the Director of CISA. Read her final in= terview [https://www.wired.com/story/big-interview-jen-easterly-cisa-cybe= rsecurity/]:

There=E2=80=99s a lot of unfinished business. We have made an impact thr=

ough our ransomware vulnerability warning pilot and our pre-ransomware not= ification initiative=2C and I=E2=80=99m really proud of that=2C because we=
work on preventing somebody from having their worst day. But ransomware i=
s still a problem. We have been laser-focused on PRC cyber actors. That wi=
ll continue to be a huge problem. I=E2=80=99m really proud of where we are=
=2C but there=E2=80=99s much=2C much more work to be done. There are thing=
s that I think we can continue driving=2C that the next administration=2C=
I hope=2C will look at=2C because=2C frankly=2C cybersecurity is a nation=
al security issue.

If Project 2025 is a guide=2C the agency will be gutted [https://www.demo= cracydocket.com/analysis/a-little-known-federal-agency-helps-secure-electi= ons-trump-wants-to-gut-it/] under Trump:

=E2=80=9CProject 2025=E2=80=99s recommendations -- essentially because t=

his one thing caused anger -- is to just strip the agency of all of its su= pport altogether=2C=E2=80=9D he said. =E2=80=9CAnd CISA=E2=80=99s function=
s go so far beyond its role in the information space in a way that would d=
o real harm to election officials and leave them less prepared to tackle f= uture challenges.=E2=80=9D

In the DHS chapter of Project 2025=2C Cucinelli suggests gutting CISA al=

most entirely=2C moving its core responsibilities on critical infrastructu=
re to the Department of Transportation. It=E2=80=99s a suggestion that Ada=
v Noti=2C the executive director of the nonpartisan voting rights advocacy=
organization Campaign Legal Center=2C previously described [https://www.= democracydocket.com/analysis/unmasking-the-anti-democracy-agenda-of-projec= t-2025/] to Democracy Docket as =E2=80=9Cabsolutely bonkers.=E2=80=9D

=E2=80=9CIt=E2=80=99s located at Homeland Security because the whole pre=

mise of the Department of Homeland Security is that it=E2=80=99s supposed=
to be the central resource for the protection of the nation=2C=E2=80=9D N=
oti said. =E2=80=9CAnd that the important functions shouldn=E2=80=99t be l= iving out in siloed agencies.=E2=80=9D

** *** ***** ******* *********** *************


** EXXONMOBIL LOBBYIST CAUGHT HACKING CLIMATE ACTIVISTS ------------------------------------------------------------

[2025.01.29] [https://www.schneier.com/blog/archives/2025/01/exxonmobil-= lobbyist-caught-hacking-climate-activists.html] The Department of Justice=
is investigating a lobbying firm representing ExxonMobil for hacking [ht= tps://www.npr.org/2025/01/24/nx-s1-5271530/hacking-investigation-climate-c= hange] the phones of climate activists:

The hacking was allegedly commissioned by a Washington=2C D.C.=2C lobbyi=

ng firm=2C according to a lawyer representing the U.S. government [https:= //legacy.www.documentcloud.org/documents/25501845-250113-usa-v-forlit/]. T=
he firm=2C in turn=2C was allegedly working on behalf of one of the world= =E2=80=99s largest oil and gas companies=2C based in Texas=2C that wanted=
to discredit groups and individuals involved in climate litigation=2C acc= ording to the lawyer for the U.S. government. In court documents=2C the Ju= stice Department does not name either company.

As part of its probe=2C the U.S. is trying to extradite an Israeli priva=

te investigator named Amit Forlit from the United Kingdom for allegedly or= chestrating the hacking campaign. A lawyer for Forlit claimed in a court f= iling [https://legacy.www.documentcloud.org/documents/25501846-usa-v-amit= -forlit-defence-skeleton-argument/] that the hacking operation her client=
is accused of leading =E2=80=9Cis alleged to have been commissioned by DC=
I Group=2C a lobbying firm representing ExxonMobil=2C one of the world=E2= =80=99s largest fossil fuel companies.=E2=80=9D

** *** ***** ******* *********** *************


** FAKE REDDIT AND WETRANSFER SITES ARE PUSHING MALWARE ------------------------------------------------------------

[2025.01.30] [https://www.schneier.com/blog/archives/2025/01/fake-reddit= -and-wetransfer-sites-are-pushing-malware.html] There are thousands of fak=
e [https://www.bleepingcomputer.com/news/security/hundreds-of-fake-reddit= -sites-push-lumma-stealer-malware/] Reddit and WeTransfer webpages that ar=
e pushing malware. They exploit people who are using search engines to sea=
rch sites like Reddit.

Unsuspecting victims clicking on the link are taken to a fake WeTransfer=

site that mimicks the interface of the popular file-sharing service. The=
=E2=80=98Download=E2=80=99 button leads to the Lumma Stealer payload [ht= tps://app.any.run/tasks/a629e4b1-433b-427e-8040-79d4aa13c245] hosted on=
=E2=80=9Cweighcobbweo[.]top.=E2=80=9D

Boing Boing post [https://boingboing.net/2025/01/28/fake-reddit-pages-are= -serving-up-malware.html].

** *** ***** ******* *********** *************


** JOURNALISTS AND CIVIL SOCIETY MEMBERS USING WHATSAPP TARGETED BY PARAGO=
N SPYWARE
------------------------------------------------------------

[2025.02.03] [https://www.schneier.com/blog/archives/2025/02/journalists= -and-civil-society-members-using-whatsapp-targeted-by-paragon-spyware.html=
] This is yet another story of commercial spyware being used against [htt= ps://www.theguardian.com/technology/2025/jan/31/whatsapp-israel-spyware] j= ournalists and civil society members.

The journalists and other civil society members were being alerted of a=

possible breach of their devices=2C with WhatsApp telling the Guardian it=
had =E2=80=9Chigh confidence=E2=80=9D that the 90 users in question had b=
een targeted and =E2=80=9Cpossibly compromised.=E2=80=9D

It is not clear who was behind the attack. Like other spyware makers=2C=

Paragon=E2=80=99s hacking software is used by government clients and What= sApp said it had not been able to identify the clients who ordered the all= eged attacks.

Experts said the targeting was a =E2=80=9Czero-click=E2=80=9D attack=2C=

which means targets would not have had to click on any malicious links to=
be infected.

** *** ***** ******* *********** *************


** DEEPFAKES AND THE 2024 US ELECTION ------------------------------------------------------------

[2025.02.04] [https://www.schneier.com/blog/archives/2025/02/deepfakes-a= nd-the-2024-us-election.html] Interesting analysis [https://knightcolumbi= a.org/blog/we-looked-at-78-election-deepfakes-political-misinformation-is-= not-an-ai-problem]:

We analyzed every instance of AI use in elections collected by the WIRED=

AI Elections Project (source [https://www.cs.princeton.edu/~sayashk/poli= tical-misinformation/WIRED-data.html] for our analysis)=2C which tracked k= nown uses of AI for creating political content during elections taking pla=
ce in 2024 worldwide. In each case=2C we identified what AI was used for a=
nd estimated the cost of creating similar content without AI.

We find that (1) half of AI use isn=E2=80=99t deceptive=2C (2) deceptive=

content produced using AI is nevertheless cheap to replicate _without_ AI=
=2C and (3) focusing on the demand for misinformation rather than the supp=
ly is a much more effective way to diagnose problems and identify interven= tions.

This tracks with my analysis. People share as a form of social signaling.=
I send you a meme/article/clipping/photo to show that we are on the same=
team. Whether it is true=2C or misinformation=2C or actual propaganda=2C=
is of secondary importance. Sometimes it=E2=80=99s completely irrelevant.=
This is why fact checking doesn=E2=80=99t work. This is why =E2=80=9Cchea=
p fakes=E2=80=9D -- obviously fake photos and videos -- are effective. Thi=
s is why=2C as the authors of that analysis said=2C the demand side is the=
real problem.

** *** ***** ******* *********** *************


** ON GENERATIVE AI SECURITY ------------------------------------------------------------

[2025.02.05] [https://www.schneier.com/blog/archives/2025/02/on-generati= ve-ai-security.html] Microsoft=E2=80=99s AI Red Team just published =E2=80= =9CLessons from Red Teaming 100 Generative AI Products [https://airedteam= whitepapers.blob.core.windows.net/lessonswhitepaper/MS_AIRT_Lessons_eBook.= pdf].=E2=80=9D Their blog post [https://www.microsoft.com/en-us/security/= blog/2025/01/13/3-takeaways-from-red-teaming-100-generative-ai-products/]=
lists =E2=80=9Cthree takeaways=2C=E2=80=9D but the eight lessons in the r= eport itself are more useful:

1. Understand what the system can do and where it is applied.
2. You don=E2=80=99t have to compute gradients to break an AI syste=

m.

3. AI red teaming is not safety benchmarking.
4. Automation can help cover more of the risk landscape.
5. The human element of AI red teaming is crucial.
6. Responsible AI harms are pervasive but difficult to measure.
7. LLMs amplify existing security risks and introduce new ones.
8. The work of securing AI systems will never be complete.

** *** ***** ******* *********** *************


** AIS AND ROBOTS SHOULD SOUND ROBOTIC ------------------------------------------------------------

[2025.02.06] [https://www.schneier.com/blog/archives/2025/02/ais-and-rob= ots-should-sound-robotic.html] Most people know that robots [https://spec= trum.ieee.org/tag/robots] no longer sound like tinny trash cans. They soun=
d like Siri [https://spectrum.ieee.org/tag/siri]=2C Alexa [https://spect= rum.ieee.org/tag/alexa]=2C and Gemini [https://spectrum.ieee.org/tag/gemi=
ni]. They sound like the voices in labyrinthine customer support phone tre=
es. And even those robot voices are being made obsolete by new AI-generate=
d voices [https://spectrum.ieee.org/chatgpt-multimodal] that can mimic ev=
ery vocal nuance and tic of human speech=2C down to specific regional acce= nts. And with just a few seconds of audio=2C AI [https://spectrum.ieee.or= g/tag/ai] can now clone someone=E2=80=99s specific voice [https://spectru= m.ieee.org/digital-afterlife].

This technology will replace humans in many areas. Automated customer supp=
ort will save money [https://spectrum.ieee.org/tag/money] by cutting staf=
fing at call centers [https://spectrum.ieee.org/tag/call-centers]. AI age=
nts [https://spectrum.ieee.org/ai-agents] will make calls on our behalf=
=2C conversing with others in natural language [https://spectrum.ieee.org= /tag/natural-language]. All of that is happening=2C and will be commonplac=
e soon.

But there is something fundamentally different about talking with a bot as=
opposed to a person. A person can be a friend. An AI cannot be a friend=
=2C despite how people might treat it or react to it. AI is at best a tool=
=2C and at worst a means of manipulation. Humans need to know whether we= =E2=80=99re talking with a living=2C breathing person or a robot with an a= genda set by the person who controls it. That=E2=80=99s why robots should=
sound like robots.

You can=E2=80=99t just label AI-generated speech. It will come in many dif= ferent forms. So we need a way to recognize AI that works no matter the mo= dality. It needs to work for long or short snippets of audio=2C even just=
a second long. It needs to work for any language=2C and in any cultural c= ontext. At the same time=2C we shouldn=E2=80=99t constrain the underlying=
system=E2=80=99s sophistication or language complexity.

We have a simple proposal: all talking AIs and robots should use a ring mo= dulator [https://spectrum.ieee.org/tag/modulator]. In the mid-twentieth c= entury=2C before it was easy to create actual robotic-sounding speech synt= hetically=2C ring modulators were used to make actors=E2=80=99 voices soun=
d robotic. Over the last few decades=2C we have become accustomed to robot=
ic voices=2C simply because text-to-speech systems were good enough to pro= duce intelligible speech that was not human-like in its sound. Now we can=
use that same technology to make robotic speech that is indistinguishable=
from human sound robotic again.

A ring modulator has several advantages: It is computationally simple=2C c=
an be applied in real-time=2C does not affect the intelligibility of the v= oice=2C and -- most importantly -- is universally =E2=80=9Crobotic soundin= g=E2=80=9D because of its historical usage for depicting robots.

Responsible AI companies [https://spectrum.ieee.org/tag/ai-companies] tha=
t provide voice synthesis [https://spectrum.ieee.org/tag/voice-synthesis]=
or AI voice assistants [https://spectrum.ieee.org/tag/voice-assistants]=
in any form should add a ring modulator of some standard frequency (say=
=2C between 30-80 Hz) and of a minimum amplitude (say=2C 20 percent). That= =E2=80=99s it. People will catch on quickly.

Here are a couple of examples you can listen to for examples of what we=E2= =80=99re suggesting. The first clip is an AI-generated =E2=80=9Cpodcast=E2= =80=9D of this article made by Google=E2=80=99s NotebookLM [https://g.co/= kgs/FyCQAGX] featuring two AI =E2=80=9Chosts.=E2=80=9D Google=E2=80=99s No= tebookLM created the podcast script and audio given only the text of this=
article. The next two clips feature that same podcast with the AIs=E2=80=
=99 voices modulated more and less subtly by a ring modulator:

* RAW AUDIO SAMPLE GENERATED BY GOOGLE=E2=80=99S NOTEBOOKLM

[https://www.schneier.com/wp-content/uploads/2025/01/robots-article.mp3]

* AUDIO SAMPLE WITH ADDED RING MODULATOR (30 HZ-25%)

[https://www.schneier.com/wp-content/uploads/2025/01/robots-article-30hz-= 25percent.mp3]

* AUDIO SAMPLE WITH ADDED RING MODULATOR (30 HZ-40%)

[https://www.schneier.com/wp-content/uploads/2025/01/robots-article-30hz-= 40percent.mp3"]

We were able to generate the audio effect with a 50-line Python [https://= spectrum.ieee.org/tag/python] script generated by Anthropic=E2=80=99s Clau=
de [https://claude.ai/]. One of the most well-known robot voices were tho=
se of the Daleks from Doctor Who [https://en.wikipedia.org/wiki/Dalek] in=
the 1960s. Back then robot voices were difficult to synthesize=2C so the=
audio was actually an actor=E2=80=99s voice run through a ring modulator.=
It was set to around 30 Hz=2C as we did in our example=2C with different=
modulation depth (amplitude) depending on how strong the robotic effect i=
s meant to be. Our expectation is that the AI industry will test and conve=
rge on a good balance of such parameters and settings=2C and will use bett=
er tools than a 50-line Python script=2C but this highlights how simple it=
is to achieve.

Of course there will also be nefarious uses of AI voices. Scams [https://= spectrum.ieee.org/tag/scams] that use voice cloning [https://spectrum.iee= e.org/tag/voice-cloning] have been getting easier every year=2C but they= =E2=80=99ve been possible for many years with the right know-how. Just lik=
e we=E2=80=99re learning that we can no longer trust images and videos we=
see because they could easily have been AI-generated=2C we will all soon=
learn that someone who sounds like a family member urgently requesting mo=
ney may just be a scammer using a voice-cloning tool.

We don=E2=80=99t expect scammers to follow our proposal: They=E2=80=99ll f=
ind a way no matter what. But that=E2=80=99s always true of security [htt= ps://spectrum.ieee.org/tag/security] standards=2C and a rising tide lifts=
all boats. We think the bulk of the uses will be with popular voice APIs=
[https://spectrum.ieee.org/tag/apis] from major companies -- and everyon=
e should know that they=E2=80=99re talking with a robot.

_This essay was written with Barath Raghavan=2C and originally appeared in=
IEEE Spectrum [https://spectrum.ieee.org/audio-deepfake-fix]._

** *** ***** ******* *********** *************


** SCREENSHOT-READING MALWARE ------------------------------------------------------------

[2025.02.07] [https://www.schneier.com/blog/archives/2025/02/screenshot-= reading-malware.html] Kaspersky is reporting [https://www.engadget.com/cy= bersecurity/kaspersky-researchers-find-screenshot-reading-malware-on-the-a= pp-store-and-google-play-211011103.html] on a new type of smartphone malwa=
re.

The malware in question uses optical character recognition (OCR) to revi=

ew a device=E2=80=99s photo library=2C seeking screenshots of recovery phr= ases for crypto wallets. Based on their assessment=2C infected Google Play=
apps have been downloaded more than 242=2C000 times. Kaspersky says: =E2= =80=9CThis is the first known case of an app infected with OCR spyware bei=
ng found in Apple=E2=80=99s official app marketplace.=E2=80=9D

That=E2=80=99s a tactic I have not heard of before.

** *** ***** ******* *********** *************


** UK IS ORDERING APPLE TO BREAK ITS OWN ENCRYPTION ------------------------------------------------------------

[2025.02.08] [https://www.schneier.com/blog/archives/2025/02/uk-is-order= ing-apple-to-break-its-own-encryption.html] The _Washington Post_ is repor= ting [https://www.washingtonpost.com/technology/2025/02/07/apple-encrypti= on-backdoor-uk/] that the UK government has served Apple with a =E2=80=9Ct= echnical capability notice=E2=80=9D as defined by the 2016 Investigatory P= owers Act=2C requiring it to break the Advanced Data Protection encryption=
in iCloud for the benefit of law enforcement.

This is a big deal=2C and something we in the security community have worr=
ied was coming for a while now.

The law=2C known by critics as the Snoopers=E2=80=99 Charter=2C makes it=

a criminal offense to reveal that the government has even made such a dem= and. An Apple spokesman declined to comment.

Apple can appeal the U.K. capability notice to a secret technical panel=

=2C which would consider arguments about the expense of the requirement=2C=
and to a judge who would weigh whether the request was in proportion to t=
he government=E2=80=99s needs. But the law does not permit Apple to delay=
complying during an appeal.

In March=2C when the company was on notice that such a requirement might=

be coming=2C it told Parliament: =E2=80=9CThere is no reason why the U.K.=
[government] should have the authority to decide for citizens of the wor=
ld whether they can avail themselves of the proven security benefits that=
flow from end-to-end encryption.=E2=80=9D

Apple is likely to turn the feature off for UK users rather than break it=
for everyone worldwide. Of course=2C UK users will be able to spoof their=
location. But this might not be enough. According to the law=2C Apple wou=
ld not be able to offer the feature to anyone who is in the UK at any poin=
t: for example=2C a visitor from the US.

And what happens next? Australia has a law [https://www.homeaffairs.gov.a= u/about-us/our-portfolios/national-security/lawful-access-telecommunicatio= ns/assistance-and-access-industry-assistance-framework] enabling it to ask=
for the same thing. Will it? Will even more countries follow?

This is madness.

** *** ***** ******* *********** *************


** PAIRWISE AUTHENTICATION OF HUMANS ------------------------------------------------------------

[2025.02.10] [https://www.schneier.com/blog/archives/2025/02/pairwise-au= thentication-of-humans.html] Here=E2=80=99s an easy [https://ksze.github.= io/PeerAuth/] system for two humans to remotely authenticate to each other=
=2C so they can be sure that neither are digital impersonations.

To mitigate that risk=2C I have developed this simple solution where you=

can setup a unique time-based one-time passcode (TOTP) between any pair o=
f persons.

This is how it works:

1. Two people=2C Person A and Person B=2C sit in front of the same=

computer and open this page;

2. They input their respective names (e.g. Alice and Bob) onto the=

same page=2C and click =E2=80=9CGenerate=E2=80=9D;

3. The page will generate two TOTP QR codes=2C one for Alice and on=

e for Bob;

4. Alice and Bob scan the respective QR code into a TOTP mobile app=

(such as Authy or Google Authenticator) on their respective mobile phones=
;

5. In the future=2C when Alice speaks with Bob over the phone or ov=

er video call=2C and wants to verify the identity of Bob=2C Alice asks Bob=
to provide the 6-digit TOTP code from the mobile app. If the code matches=
what Alice has on her own phone=2C then Alice has more confidence that sh=
e is speaking with the real Bob.

Simple=2C and clever.

** *** ***** ******* *********** *************


** TRUSTED EXECUTION ENVIRONMENTS ------------------------------------------------------------

[2025.02.11] [https://www.schneier.com/blog/archives/2025/02/trusted-enc= ryption-environments.html] Really good -- and detailed -- survey [https:/= /dl.acm.org/doi/pdf/10.1145/3634737.3644993] of Trusted Execution Environm= ents (TEEs.)

** *** ***** ******* *********** *************


** DELIVERING MALWARE THROUGH ABANDONED AMAZON S3 BUCKETS ------------------------------------------------------------

[2025.02.12] [https://www.schneier.com/blog/archives/2025/02/delivering-= malware-through-abandoned-amazon-s3-buckets.html] Here=E2=80=99s a supply-= chain attack [https://labs.watchtowr.com/8-million-requests-later-we-made= -the-solarwinds-supply-chain-attack-look-amateur/] just waiting to happen.=
A group of researchers searched for=2C and then registered=2C abandoned A= mazon S3 buckets for about $400. These buckets contained software librarie=
s that are still used. Presumably the projects don=E2=80=99t realize that=
they have been abandoned=2C and still ping them for patches=2C updates=2C=
and etc.

The TL;DR is that this time=2C we ended up discovering ~150 Amazon S3 bu=

ckets that had previously been used across commercial and open source soft= ware products=2C governments=2C and infrastructure deployment/update pipel= ines -- and then abandoned.

Naturally=2C we registered them=2C just to see what would happen -- =E2=

=80=9Dhow many people are really trying to request software updates from S=
3 buckets that appear to have been abandoned months or even years ago?=E2= =80=9D=2C we naively thought to ourselves.

Turns out they got eight million requests over two months.

Had this been an actual attack=2C they would have modified the code in tho=
se buckets to contain malware and watch as it was incorporated in differen=
t software builds around the internet. This is basically the SolarWinds at= tack=2C but much more extensive.

But there=E2=80=99s a second dimension to this attack. Because these updat=
e buckets are abandoned=2C the developers who are using them also no longe=
r have the power to patch them automatically to protect them. The mechanis=
m they would use to do so is now in the hands of adversaries. Moreover=2C=
often -- but not always -- losing the bucket that they=E2=80=99d use for=
it also removes the original vendor=E2=80=99s ability to identify the vul= nerable software in the first place. That hampers their ability to communi= cate with vulnerable installations.

Software supply-chain security is an absolute mess. And it=E2=80=99s not g= oing to be easy=2C or cheap=2C to fix. Which means that it won=E2=80=99t b=
e. Which is an even worse mess.

** *** ***** ******* *********** *************


** DOGE AS A NATIONAL CYBERATTACK ------------------------------------------------------------

[2025.02.13] [https://www.schneier.com/blog/archives/2025/02/doge-as-a-n= ational.html] In the span of just weeks=2C the US government has experienc=
ed what may be the most consequential security breach in its history -- no=
t through a sophisticated cyberattack or an act of foreign espionage=2C bu=
t through official orders by a billionaire with a poorly defined governmen=
t role. And the implications for national security are profound.

First=2C it was reported that people associated with the newly created Dep= artment of Government Efficiency (DOGE) had accessed [https://bsky.app/pr= ofile/wyden.senate.gov/post/3lh5ejpwncc23] the [https://www.nytimes.com/2= 025/02/01/us/politics/elon-musk-doge-federal-payments-system.html] US [ht= tps://nymag.com/intelligencer/article/elon-musk-doge-treasury-access-feder= al-payments.html] Treasury [https://therecord.media/union-groups-sue-trea= sury-over-giving-doge-access-to-data] computer system=2C giving them the a= bility to collect data on and potentially control the department=E2=80=99s=
roughly $5.45 trillion [https://fiscal.treasury.gov/fds/] in annual fede=
ral payments.

Then=2C we learned that uncleared DOGE personnel had gained access to clas= sified [https://www.nbcnews.com/politics/national-security/usaid-security= -leaders-removed-refusing-elon-musks-doge-employees-acce-rcna190357] data=
from the US Agency for International Development=2C possibly copying it o=
nto their own systems. Next=2C the Office of Personnel Management -- which=
holds detailed personal data on millions of federal employees=2C includin=
g those with security clearances -- was [https://fedscoop.com/opm-email-f= ederal-workforce-lawsuit-server-privacy-security/] compromised [https://w= ww.yahoo.com/tech/elon-musk-seizes-computer-system-171738117.html]. After=
that=2C Medicaid and Medicare records [https://www.reuters.com/world/us/= doge-aides-search-medicare-agency-payment-systems-fraud-wsj-reports-2025-0= 2-05/] were compromised.

Meanwhile=2C only partially redacted names of CIA employees were sent [ht= tps://thehill.com/policy/national-security/5129170-cia-email-employee-iden= tities/] over an unclassified email account. DOGE personnel are also repor=
ted to be feeding [https://www.washingtonpost.com/nation/2025/02/06/elon-= musk-doge-ai-department-education/] Education Department data into artific=
ial intelligence software=2C and they have also started working [https://= www.reuters.com/world/us/three-doge-members-raise-access-concerns-us-energ= y-department-sources-say-2025-02-07/] at the Department of Energy.

This story is moving very fast. On Feb. 8=2C a federal judge blocked [htt= ps://www.reuters.com/legal/us-judge-temporarily-blocks-musks-doge-accessin= g-payment-systems-2025-02-08/] the DOGE team from accessing the Treasury D= epartment systems any further. But given that DOGE workers have already co= pied data and possibly installed and modified software=2C it=E2=80=99s unc= lear how this fixes anything.

In any case=2C breaches of other critical government systems are likely to=
follow unless federal employees stand firm on the protocols protecting na= tional security.

The systems that DOGE is accessing are not esoteric pieces of our nation= =E2=80=99s infrastructure -- they are the sinews of government [https://w= ww.lawfaremedia.org/article/elon-musk-weaponizes-the-government].

For example=2C the Treasury Department systems contain the technical bluep= rints for how the federal government moves money=2C while the Office of Pe= rsonnel Management (OPM) network contains information on who and what orga= nizations the government employs and contracts with.

What makes this situation unprecedented isn=E2=80=99t just the scope=2C bu=
t also the method of attack. Foreign adversaries typically spend years att= empting to penetrate government systems such as these=2C using stealth to=
avoid being seen and carefully hiding any tells or tracks. The Chinese go= vernment=E2=80=99s 2015 breach of OPM [https://www.washingtonpost.com/wor= ld/national-security/chinese-hackers-breach-federal-governments-personnel-= office/2015/06/04/889c0e52-0af7-11e5-95fd-d580f1c5d44e_story.html] was a s= ignificant US security failure=2C and it illustrated how personnel data co=
uld be used to identify intelligence officers and compromise national secu= rity.

In this case=2C external operators with limited experience [https://www.w= ired.com/story/elon-musk-government-young-engineers/] and minimal oversigh=
t are doing their work in plain sight and under massive public scrutiny: g= aining the highest levels of administrative access [https://talkingpoints= memo.com/edblog/musk-cronies-dive-into-treasury-dept-payments-code-base] a=
nd making changes to the United States=E2=80=99 most sensitive networks=2C=
potentially introducing new security vulnerabilities in the process.

But the most alarming aspect isn=E2=80=99t just the access being granted.=
It=E2=80=99s the systematic dismantling of security measures that would d= etect and prevent misuse -- including standard incident response protocols=
=2C auditing=2C and change-tracking mechanisms -- by [https://www.theguar= dian.com/us-news/2025/feb/02/usaid-officials-put-on-leave-musk-doge] remov=
ing the career officials in charge of those security measures and replacin=
g them with inexperienced operators.

The Treasury=E2=80=99s computer systems have such an impact on national se= curity that they were designed with the same principle that guides nuclear=
launch protocols: No single person should have unlimited power. Just as l= aunching a nuclear missile requires two separate officers turning their ke=
ys simultaneously=2C making changes to critical financial systems traditio= nally requires multiple authorized personnel working in concert.

This approach=2C known as =E2=80=9Cseparation of duties=2C=E2=80=9D isn=E2= =80=99t just bureaucratic red tape; it=E2=80=99s a fundamental security pr= inciple as old as banking itself. When your local bank processes a large t= ransfer=2C it requires two different employees to verify the transaction.=
When a company issues a major financial report=2C separate teams must rev=
iew and approve it. These aren=E2=80=99t just formalities -- they=E2=80=99=
re essential safeguards against corruption and error. These measures have=
been bypassed or ignored [https://www.wired.com/story/elon-musk-governme= nt-young-engineers/]. It=E2=80=99s as if someone found a way to rob Fort K=
nox by simply declaring that the new official policy is to fire all the gu= ards and allow unescorted visits to the vault.

The implications for national security are staggering [https://techcrunch= =2Ecom/2025/02/05/the-biggest-breach-of-u-s-government-data-is-under-way/].=
Sen. Ron Wyden said his office had learned that the attackers gained priv= ileges [https://securityaffairs.com/173776/security/elon-musk-s-doge-gran= ted-full-access-to-sensitive-treasury-systems.html] that allow them to mod=
ify core programs in Treasury Department computers that verify federal pay= ments=2C access encrypted keys that secure financial transactions=2C and a= lter audit logs that record system changes. Over at OPM=2C reports indicat=
e that individuals associated with DOGE connected [https://www.rawstory.c= om/elon-musk-doge-lawsuit/] an unauthorized server into the network. They=
are also reportedly training [https://gizmodo.com/elon-musks-doge-runnin= g-highly-sensitive-government-data-through-ai-report-2000560381] AI [http= s://www.washingtonpost.com/nation/2025/02/06/elon-musk-doge-ai-department-= education/] software on all of this sensitive data.

This is much more critical than the initial unauthorized access. These new=
servers have unknown capabilities and configurations=2C and there=E2=80=
=99s no evidence that this new code has gone through any rigorous security=
testing protocols. The AIs being trained are certainly not secure enough=
for this kind of data. All are ideal targets for any adversary=2C foreign=
or domestic=2C also seeking access to federal data.

There=E2=80=99s a reason why every modification -- hardware or software --=
to these systems goes through a complex planning process and includes sop= histicated access-control mechanisms. The national security crisis is that=
these systems are now much more vulnerable to dangerous attacks at the sa=
me time that the legitimate system administrators trained to protect them=
have been locked out [https://www.reuters.com/world/us/musk-aides-lock-g= overnment-workers-out-computer-systems-us-agency-sources-say-2025-01-31/].

By modifying core systems=2C the attackers have not only compromised curre=
nt operations=2C but have also left behind vulnerabilities that could be e= xploited in future attacks -- giving adversaries such as Russia and China=
an unprecedented [https://therecord.media/doge-opm-treasury-cybersecurit=
y] opportunity [https://cyberscoop.com/musk-doge-opm-treasury-breach/]. T=
hese countries have long targeted these systems. And they don=E2=80=99t ju=
st want to gather intelligence -- they also want to understand how to disr=
upt these systems in a crisis.

Now=2C the technical details of how these systems operate=2C their securit=
y protocols=2C and their vulnerabilities are now potentially exposed to un= known parties without any of the usual safeguards. Instead of having to br= each heavily fortified digital walls=2C these parties can simply walk thr= ough doors that are being propped open -- and then erase evidence of their=
actions.

The security implications span three critical areas.

First=2C system manipulation: External operators can now modify operations=
while also altering audit trails that would track their changes. Second=
=2C data exposure: Beyond accessing personal information and transaction r= ecords=2C these operators can copy entire system architectures and securit=
y configurations -- in one case=2C the technical blueprint of the country= =E2=80=99s federal payment infrastructure. Third=2C and most critically=2C=
is the issue of system control: These operators can alter core systems an=
d authentication mechanisms while disabling the very tools designed to det=
ect such changes. This is more than modifying operations; it is modifying=
the infrastructure that those operations use.

To address these vulnerabilities=2C three immediate steps are essential. F= irst=2C unauthorized access must be revoked and proper authentication prot= ocols restored. Next=2C comprehensive system monitoring and change managem=
ent must be reinstated -- which=2C given the difficulty of cleaning a comp= romised system=2C will likely require a complete system reset. Finally=2C=
thorough audits must be conducted of all system changes made during this=
period.

This is beyond politics -- this is a matter of national security. Foreign=
national intelligence organizations will be quick to take advantage of bo=
th the chaos and the new insecurities to steal US data and install backdoo=
rs to allow for future access.

Each day of continued unrestricted access makes the eventual recovery more=
difficult and increases the risk of irreversible damage to these critical=
systems. While the full impact may take time to assess=2C these steps rep= resent the minimum necessary actions to begin restoring system integrity a=
nd security protocols.

Assuming that anyone in the government still cares.

_This essay was written with Davi Ottenheimer=2C and originally appeared i=
n Foreign Policy [https://foreignpolicy.com/2025/02/11/doge-cyberattack-u= nited-states-treasury/]._

** *** ***** ******* *********** *************


** AI AND CIVIL SERVICE PURGES ------------------------------------------------------------

[2025.02.14] [https://www.schneier.com/blog/archives/2025/02/ai-and-civi= l-service-purges.html] Donald Trump and Elon Musk=E2=80=99s chaotic approa=
ch to reform is upending government operations. Critical functions have be=
en halted [https://www.theguardian.com/us-news/2025/feb/05/musk-doge-take= over-usaid]=2C tens of thousands of federal staffers are being encouraged=
to resign [https://www.wsj.com/lifestyle/careers/federal-workers-accept-= buyout-offers-be1c00fb]=2C and congressional mandates are being disregarde=
d [https://thehill.com/business/5124133-democrats-bill-treasury-system-mu= sk/]. The next phase: The Department of Government Efficiency reportedly [= https://www.nytimes.com/2025/02/03/technology/musk-allies-ai-government.h=
tml] wants to use AI to cut costs. According to _The Washington Post_=2C M= usk=E2=80=99s group has started to run sensitive data [https://www.washin= gtonpost.com/nation/2025/02/06/elon-musk-doge-ai-department-education/] fr=
om government systems through AI programs to analyze spending and determin=
e what could be pruned. This may lead to the elimination of human jobs in=
favor of automation. As one government official who has been tracking Mus= k=E2=80=99s DOGE team told the_ Post_=2C the ultimate aim is to use AI to=
replace =E2=80=9Cthe human workforce with machines [https://www.washingt= onpost.com/business/2025/02/08/doge-musk-goals/].=E2=80=9D (Spokespeople f=
or the White House and DOGE did not respond to requests for comment.)

Using AI to make government more efficient is a worthy pursuit=2C and this=
is not a new idea. The Biden administration disclosed more than 2=2C000 A=
I applications [https://github.com/ombegov/2024-Federal-AI-Use-Case-Inven= tory] in development across the federal government. For example=2C FEMA [= https://www.dhs.gov/ai/use-case-inventory/fema] has started using AI to he=
lp perform damage assessment in disaster areas. The Centers for Medicare a=
nd Medicaid Services [https://ai.cms.gov/assets/CMS_AI_Playbook.pdf] has=
started using AI to look for fraudulent billing. The idea of replacing de= dicated and principled civil servants with AI agents=2C however=2C _is _ne=
w -- and complicated.

The civil service -- the massive cadre of employees who operate government=
agencies -- plays a vital role in translating laws and policy into the op= eration of society. New presidents can issue sweeping executive orders=2C=
but they often have no real effect until they actually change the behavio=
r of public servants. Whether you think of these people as essential and i= nspiring [https://www.washingtonpost.com/opinions/interactive/2024/michae= l-lewis-conclusion-who-is-government/] do-gooders=2C boring bureaucratic f= unctionaries=2C or as agents of a =E2=80=9Cdeep state [https://www.theatl= antic.com/health/archive/2024/11/deep-state-public-health-trump-kennedy/68= 0621/]=2C=E2=80=9D their sheer number and continuity act as ballast that r= esists institutional change.

This is why Trump and Musk=E2=80=99s actions are so significant. The more=
AI decision making is integrated into government=2C the easier change wil=
l be. If human workers are widely replaced with AI=2C executives will have=
unilateral authority to instantaneously alter the behavior of the governm= ent=2C profoundly raising the stakes for transitions of power in democracy=
=2E Trump=E2=80=99s unprecedented purge of the civil service might be the la= st time a president needs to replace the human beings in government in ord=
er to dictate its new functions. Future leaders may do so at the press of=
a button.

To be clear=2C the use of AI by the executive branch doesn=E2=80=99t have=
to be disastrous. In theory=2C it could allow new leadership to swiftly i= mplement the wishes of its electorate. But this could go very badly in the=
hands of an authoritarian leader. AI systems concentrate power at the top=
=2C so they could allow an executive to effectuate change over sprawling b= ureaucracies instantaneously. Firing and replacing tens of thousands of hu=
man bureaucrats is a huge undertaking. Swapping one AI out for another=2C=
or modifying the rules that those AIs operate by=2C would be much simpler=
=2E

Social-welfare programs=2C if automated with AI=2C could be redirected to=
systematically benefit one group and disadvantage another with a single p= rompt change. Immigration-enforcement agencies could prioritize people for=
investigation and detainment with one instruction. Regulatory-enforcement=
agencies that monitor corporate behavior for malfeasance could turn their=
attention to=2C or away from=2C any given company on a whim.

Even if Congress were motivated to fight back against Trump and Musk=2C or=
against a future president seeking to bulldoze the will of the legislatur= e=2C the absolute power to command AI agents would make it easier to subve=
rt legislative intent. AI has the power to diminish [https://www.techpoli= cy.press/anatomy-of-an-ai-coup/] representative politics. Written law is n= ever fully determinative of the actions of government -- there is always w= iggle room for presidents=2C appointed leaders=2C and civil servants to ex= ercise their own judgment. Whether intentional or not=2C whether charitabl=
y or not=2C each of these actors uses discretion. In human systems=2C that=
discretion is widely distributed across many individuals -- people who=2C=
in the case of career civil servants=2C usually outlast presidencies.

Today=2C the AI ecosystem is dominated by a small number of corporations t=
hat decide how the most widely used AI models are designed=2C which data t=
hey are trained on=2C and which instructions they follow. Because their wo=
rk is largely secretive and unaccountable [https://crfm.stanford.edu/fmti= /paper.pdf] to public interest=2C these tech companies are capable of maki=
ng changes to the bias of AI systems -- either generally or with aim at sp= ecific governmental use cases -- that are invisible to the rest of us. And=
these private actors are both vulnerable to coercion by political leaders=
and self-interested in appealing to their favor. Musk himself created and=
funded xAI=2C now one of the world=E2=80=99s largest AI labs=2C with an e= xplicitly ideological [https://www.zdnet.com/article/i-tried-xs-anti-woke= -grok-ai-chatbot-the-results-were-the-opposite-of-what-i-expected/] mandat=
e to generate anti-=E2=80=9Cwoke=E2=80=9D AI and steer [https://www.wired= =2Ecom/llm-political-bias/] the wider AI industry in a similar direction.

But there=E2=80=99s a second way that AI=E2=80=99s transformation of gover= nment could go. AI development could happen inside of transparent and acco= untable public institutions=2C alongside its continued development by Big=
Tech. Applications of AI in democratic governments could be focused on be= nefitting public servants and the communities they serve by=2C for example=
=2C making it easier for non-English speakers to access government service= s=2C making ministerial tasks such as processing routine applications more=
efficient and reducing backlogs=2C or helping constituents weigh in on th=
e policies deliberated by their representatives. Such AI integrations shou=
ld be done gradually and carefully=2C with public oversight for their desi=
gn and implementation and monitoring and guardrails to avoid unacceptable=
bias and harm.

Governments around the world are demonstrating how this could be done=2C t= hough it=E2=80=99s early days. Taiwan [https://talktothecity.org] has pio= neered the use of AI models to facilitate deliberative democracy at an unp= recedented scale. Singapore has been a leader in the development of public=
AI [https://www.brookings.edu/articles/how-public-ai-can-strengthen-demo= cracy/] models=2C built transparently [https://sea-lion.ai] and with publ= ic-service use cases [https://www.undp.org/policy-centre/singapore/blog/p= airing-ai-public-sector-impact-singapore] in mind. Canada [https://www.ca= nada.ca/en/government/system/digital-government/digital-government-innovat= ions/responsible-use-ai/algorithmic-impact-assessment.html] has illustrate=
d the role of disclosure and public input on the consideration of AI use c= ases in government. Even if you do not trust the current White House to fo= llow any of these examples=2C U.S. states -- which have much greater conta=
ct and influence over the daily lives of Americans than the federal govern= ment -- could lead the way on this kind of responsible development and dep= loyment of AI.

As the political theorist David Runciman [https://wwnorton.com/books/9781= 631496943] has written=2C AI is just another in a long line of artificial=
=E2=80=9Cmachines=E2=80=9D used to govern how people live and act=2C not=
unlike corporations and states before it. AI doesn=E2=80=99t replace thos=
e older institutions=2C but it changes how they function. As the Trump adm= inistration forges stronger ties to Big Tech and AI developers=2C we need=
to recognize the potential of that partnership to steer the future of dem= ocratic governance -- and act to make sure that it does not enable future=
authoritarians.

_This essay was written with Nathan E. Sanders=2C and originally appeared=
in The Atlantic [https://www.theatlantic.com/technology/archive/2025/02/= doge-ai-plans/681635/]._

** *** ***** ******* *********** *************


** UPCOMING SPEAKING ENGAGEMENTS ------------------------------------------------------------

[2025.02.14] [https://www.schneier.com/blog/archives/2025/02/upcoming-sp= eaking-engagements-43.html] This is a current list of where and when I am=
scheduled to speak:

* I=E2=80=99m speaking at Boskone 62 [https://boskone.org/] in Bosto=
n=2C Massachusetts=2C USA=2C which runs from February 14-16=2C 2025. My ta=
lk is at 4:00 PM ET on the 15th.
* I=E2=80=99m speaking at the Rossfest Symposium [https://www.cl.cam= =2Eac.uk/events/rossfest/] in Cambridge=2C UK=2C on March 25=2C 2025.

The list is maintained on this page [https://www.schneier.com/events/].

** *** ***** ******* *********** *************

Since 1998=2C CRYPTO-GRAM has been a free monthly newsletter providing sum= maries=2C analyses=2C insights=2C and commentaries on security technology.=
To subscribe=2C or to read back issues=2C see Crypto-Gram's web page [ht= tps://www.schneier.com/crypto-gram/].

You can also read these articles on my blog=2C Schneier on Security [http= s://www.schneier.com].

Please feel free to forward CRYPTO-GRAM=2C in whole or in part=2C to colle= agues and friends who will find it valuable. Permission is also granted to=
reprint CRYPTO-GRAM=2C as long as it is reprinted in its entirety.

Bruce Schneier is an internationally renowned security technologist=2C cal=
led a security guru by the _Economist_. He is the author of over one dozen=
books -- including his latest=2C _A Hacker=E2=80=99s Mind_ [https://www.= schneier.com/books/a-hackers-mind/] -- as well as hundreds of articles=2C=
essays=2C and academic papers. His newsletter and blog are read by over 2= 50=2C000 people. Schneier is a fellow at the Berkman Klein Center for Inte= rnet & Society at Harvard University; a Lecturer in Public Policy at the H= arvard Kennedy School; a board member of the Electronic Frontier Foundatio= n=2C AccessNow=2C and the Tor Project; and an Advisory Board Member of the=
Electronic Privacy Information Center and VerifiedVoting.org. He is the C= hief of Security Architecture at Inrupt=2C Inc.

Copyright (c) 2025 by Bruce Schneier.

** *** ***** ******* *********** *************

Mailing list hosting graciously provided by MailChimp [https://mailchimp.= com/]. Sent without web bugs or link tracking.

This email was sent to: cryptogram@toolazy.synchro.net

_You are receiving this email because you subscribed to the Crypto-Gram ne= wsletter._

Unsubscribe from this list: https://schneier.us18.list-manage.com/unsubscr= ibe?u=3Df99e2b5ca82502f48675978be&id=3D22184111ab&t=3Db&e=3D70f249ec14&c=3D0= ce3e98ce0

Update subscription preferences: https://schneier.us18.list-manage.com/pro= file?u=3Df99e2b5ca82502f48675978be&id=3D22184111ab&e=3D70f249ec14&c=3D0ce3e9= 8ce0

Bruce Schneier
Harvard Kennedy School
1 Brattle Square
Cambridge=2C MA 02138
USA
--_----------=_MCPart_79213603
Content-Type: text/html; charset="utf-8"
Content-Transfer-Encoding: quoted-printable

<!DOCTYPE html><html lang=3D"en"><head><meta charset=3D"UTF-8"><title>Cryp= to-Gram=2C February 15=2C 2025</title></head><body>
<div class=3D"preview-text" style=3D"display:none !important;mso-hide:all;= font-size:1px;line-height:1px;max-height:0px;max-width:0px;opacity:0;overf= low:hidden;">A monthly newsletter about cybersecurity and related topics.<= /div>
<h1 style=3D"font-size:140%">Crypto-Gram <br>
<span style=3D"display:block;padding-top:.5em;font-size:80%">February 15=
=2C 2025</span></h1>


<p>by Bruce Schneier
<br>Fellow and Lecturer=2C Harvard Kennedy School
<br>schneier@schneier.com
<br><a href=3D"https://www.schneier.com">https://www.schneier.com</a>


<p>A free monthly newsletter providing summaries=2C analyses=2C insights=
=2C and commentaries on security: computer and otherwise.</p>

<p>For back issues=2C or to subscribe=2C visit <a href=3D"https://www.schn= eier.com/crypto-gram/">Crypto-Gram's web page</a>.</p>

<p><a href=3D"https://www.schneier.com/crypto-gram/archives/2025/0215.html= ">Read this issue on the web</a></p>

<p>These same essays and news items appear in the <a href=3D"https://www.s= chneier.com/">Schneier on Security</a> blog=2C along with a lively and int= elligent comment section. An RSS feed is available.</p>

<p style=3D"font-size:88%">** *** ***** ******* *********** *************<=


<h2 style=3D"font-size:125%;font-weight:bold" id=3D"toc"><a name=3D"toc">I=
n this issue:</a></h2>

<p><em>If these links don't work in your email client=2C try <a href=3D"ht= tps://www.schneier.com/crypto-gram/archives/2025/0215.html">reading this i= ssue of Crypto-Gram on the web.</a></em></p>




<li><a href=3D"#cg1">Phishing False Alarm</a></li>
<li><a href=3D"#cg2">FBI Deletes PlugX Malware from Thousands of Computers= </a></li>
<li><a href=3D"#cg3">Social Engineering to Disable iMessage Protections</a= ></li>
<li><a href=3D"#cg4">Biden Signs New Cybersecurity Order</a></li>
<li><a href=3D"#cg5">AI Mistakes Are Very Different from Human Mistakes</a= ></li>
<li><a href=3D"#cg6">AI Will Write Complex Laws</a></li>
<li><a href=3D"#cg7">Third Interdisciplinary Workshop on Reimagining Democ= racy (IWORD 2024)</a></li>
<li><a href=3D"#cg8">New VPN Backdoor</a></li>
<li><a href=3D"#cg9">CISA Under Trump</a></li>
<li><a href=3D"#cg10">ExxonMobil Lobbyist Caught Hacking Climate Activists= </a></li>
<li><a href=3D"#cg11">Fake Reddit and WeTransfer Sites Are Pushing Malware= </a></li>
<li><a href=3D"#cg12">Journalists and Civil Society Members Using WhatsApp=
Targeted by Paragon Spyware</a></li>
<li><a href=3D"#cg13">Deepfakes and the 2024 US Election</a></li>
<li><a href=3D"#cg14">On Generative AI Security</a></li>
<li><a href=3D"#cg15">AIs and Robots Should Sound Robotic</a></li>
<li><a href=3D"#cg16">Screenshot-Reading Malware</a></li>
<li><a href=3D"#cg17">UK Is Ordering Apple to Break Its Own Encryption</a>= </li>
<li><a href=3D"#cg18">Pairwise Authentication of Humans</a></li>
<li><a href=3D"#cg19">Trusted Execution Environments</a></li>
<li><a href=3D"#cg20">Delivering Malware Through Abandoned Amazon S3 Bucke= ts</a></li>
<li><a href=3D"#cg21">DOGE as a National Cyberattack</a></li>
<li><a href=3D"#cg22">AI and Civil Service Purges</a></li>
<li><a href=3D"#cg23">Upcoming Speaking Engagements</a></li>
</ol>

<p style=3D"font-size:88%">** *** ***** ******* *********** *************<=


<h2 style=3D"font-size:125%;font-weight:bold" id=3D"cg1"><a name=3D"cg1">P= hishing False Alarm</a></h2>

<p><a href=3D"https://www.schneier.com/blog/archives/2025/01/phishing-fals= e-alarm.html"><strong>[2025.01.15]</strong></a> A very security-conscious=
company was hit with a (presumed) massive state-actor phishing attack wit=
h gift cards=2C and everyone rallied to combat it -- until it <a href=3D"h= ttps://notalwaysright.com/?p=3D359144">turned out</a> it was company manag= ement sending the gift cards.</p>

<p style=3D"font-size:88%">** *** ***** ******* *********** *************<=


<h2 style=3D"font-size:125%;font-weight:bold" id=3D"cg2"><a name=3D"cg2">F=
BI Deletes PlugX Malware from Thousands of Computers</a></h2>

<p><a href=3D"https://www.schneier.com/blog/archives/2025/01/fbi-deletes-p= lugx-malware-from-thousands-of-computers.html"><strong>[2025.01.16]</stro= ng></a> According to a DOJ <a href=3D"https://www.justice.gov/opa/pr/justi= ce-department-and-fbi-conduct-international-operation-delete-malware-used-= china-backed">press release</a>=2C the FBI was able to delete the Chinese-= used PlugX malware from =E2=80=9Capproximately 4=2C258 U.S.-based computer=
s and networks.=E2=80=9D</p>

<p><a href=3D"https://gizmodo.com/the-fbi-says-it-made-malware-delete-itse= lf-from-americans-computers-2000550046">Details</a>:</p>

<blockquote><p>To retrieve information from and send commands to the hacke=
d machines=2C the malware connects to a command-and-control server that is=
operated by the hacking group. <a href=3D"https://www.justice.gov/opa/med= ia/1384136/dl">According to the FBI</a>=2C at least 45=2C000 IP addresses=
in the US had back-and-forths with the command-and-control server since S= eptember 2023.</p>

<p>It was that very server that allowed the FBI to finally kill this pesky=
bit of malicious software. First=2C they tapped the know-how of French in= telligence agencies=2C which had <a href=3D"https://www.bleepingcomputer.c= om/news/security/french-police-push-plugx-malware-self-destruct-payload-to= -clean-pcs/">recently discovered a technique</a> for getting PlugX to self= -destruct. Then=2C the FBI gained access to the hackers=E2=80=99 command-a= nd-control server and used it to request all the IP addresses of machines=
that were actively infected by PlugX. Then it sent a command via the serv=
er that causes PlugX to delete itself from its victims=E2=80=99 computers.= </p></blockquote>

<p style=3D"font-size:88%">** *** ***** ******* *********** *************<=


<h2 style=3D"font-size:125%;font-weight:bold" id=3D"cg3"><a name=3D"cg3">S= ocial Engineering to Disable iMessage Protections</a></h2>

<p><a href=3D"https://www.schneier.com/blog/archives/2025/01/social-engine= ering-to-disable-imessage-protections.html"><strong>[2025.01.17]</strong>=
</a> I am always interested in new phishing tricks=2C and watching them sp=
read across the ecosystem.</p>

<p>A few days ago I started getting phishing SMS messages with a new twist=
=2E They were standard messages about delayed packages or somesuch=2C with t= he goal of getting me to click on a link and entering some personal inform= ation into a website. But because they came from unknown phone numbers=2C=
the links did not work. So -- this is the new bit -- the messages said so= mething like: =E2=80=9CPlease reply Y=2C then exit the text message=2C reo=
pen the text message activation link=2C or copy the link to Safari browser=
to open it.=E2=80=9D</p>

<p>I saw it once=2C and now I am seeing it again and again. Everyone has n=
ow adopted this new trick.</p>

<p>One article claims that this trick has been popular since <a href=3D"ht= tps://www.bleepingcomputer.com/news/security/phishing-texts-trick-apple-im= essage-users-into-disabling-protection/">last summer</a>. I don=E2=80=99t=
know; I would have expected to have seen it before last weekend.</p>

<p style=3D"font-size:88%">** *** ***** ******* *********** *************<=


<h2 style=3D"font-size:125%;font-weight:bold" id=3D"cg4"><a name=3D"cg4">B= iden Signs New Cybersecurity Order</a></h2>

<p><a href=3D"https://www.schneier.com/blog/archives/2025/01/biden-signs-n= ew-cybersecurity-order.html"><strong>[2025.01.20]</strong></a> President=
Biden has signed a <a href=3D"https://bidenwhitehouse.archives.gov/briefi= ng-room/presidential-actions/2025/01/16/executive-order-on-strengthening-a= nd-promoting-innovation-in-the-nations-cybersecurity/">new cybersecurity o= rder</a>. It has a bunch of provisions=2C most notably using the US govern= ment=E2=80=99s procurement power to improve cybersecurity practices indust= ry-wide.</p>

<p>Some <a href=3D"https://www.wired.com/story/biden-executive-order-cyber= security-ai-and-more/">details</a>:</p>

<blockquote><p>The core of the executive order is an array of mandates for=
protecting government networks based on lessons learned from recent major=
incidents -- namely=2C the security failures of federal contractors.</p>

<p>The order requires software vendors to submit proof that they follow se= cure development practices=2C building on <a href=3D"https://www.whitehous= e.gov/wp-content/uploads/2022/09/M-22-18.pdf">a mandate that debuted</a> i=
n 2022 in response to <a href=3D"https://www.whitehouse.gov/briefing-room/= presidential-actions/2021/05/12/executive-order-on-improving-the-nations-c= ybersecurity/">Biden=E2=80=99s first cyber executive order</a>. The Cybers= ecurity and Infrastructure Security Agency would be tasked with double-che= cking these security attestations and working with vendors to fix any prob= lems. To put some teeth behind the requirement=2C the White House=E2=80=99=
s Office of the National Cyber Director is =E2=80=9Cencouraged to refer at= testations that fail validation to the Attorney General=E2=80=9D for poten= tial investigation and prosecution.</p>

<p>The order gives the Department of Commerce eight months to assess the m=
ost commonly used cyber practices in the business community and issue guid= ance based on them. Shortly thereafter=2C those practices would become man= datory for companies seeking to do business with the government. The direc= tive also kicks off updates to the National Institute of Standards and Tec= hnology=E2=80=99s <a href=3D"https://csrc.nist.gov/projects/ssdf">secure s= oftware development guidance.</a></p></blockquote>

<p><a href=3D"https://therecord.media/biden-cybersecurity-executive-order"= >More</a> <a href=3D"https://www.csoonline.com/article/3802476/biden-white= -house-to-go-all-out-in-final-sweeping-cybersecurity-order.html">informati= on</a>.</p>

<p style=3D"font-size:88%">** *** ***** ******* *********** *************<=


<h2 style=3D"font-size:125%;font-weight:bold" id=3D"cg5"><a name=3D"cg5">A=
I Mistakes Are Very Different from Human Mistakes</a></h2>

<p><a href=3D"https://www.schneier.com/blog/archives/2025/01/ai-mistakes-a= re-very-different-from-human-mistakes.html"><strong>[2025.01.21]</strong>=
</a> Humans make mistakes all the time. All of us do=2C every day=2C in ta=
sks both new and routine. Some of our mistakes are minor and some are cata= strophic. Mistakes can break trust with our friends=2C lose the confidence=
of our bosses=2C and sometimes be the difference between life and death.<=


<p>Over the millennia=2C we have created security systems to deal with the=
sorts of mistakes humans commonly make. These days=2C casinos rotate thei=
r dealers regularly=2C because they make mistakes if they do the same task=
for too long. Hospital personnel write on limbs before surgery so that do= ctors operate on the correct body part=2C and they count surgical instrume=
nts to make sure none were left inside the body. From copyediting to doubl= e-entry bookkeeping to appellate courts=2C we humans have gotten really go=
od at correcting human mistakes.</p>

<p>Humanity is now rapidly integrating a wholly different kind of mistake-= maker into society: AI. Technologies like <a href=3D"https://spectrum.ieee= =2Eorg/tag/llms">large language models</a> (LLMs) can perform many cognitive=
tasks traditionally fulfilled by humans=2C but they make plenty of mistak=
es. It seems <a href=3D"https://www.buzzfeed.com/carleysuthers/weird-and-w= rong-ai-responses">ridiculous</a> when chatbots tell you to eat rocks or a=
dd glue to pizza. But it=E2=80=99s not the frequency or severity of AI sys= tems=E2=80=99 mistakes that differentiates them from human mistakes. It=E2= =80=99s their weirdness. AI systems do not make mistakes in the same ways=
that humans do.</p>

<p>Much of the friction -- and risk -- associated with our use of AI arise=
from that difference. We need to invent new <a href=3D"https://spectrum.i= eee.org/tag/security">security</a> systems that adapt to these differences=
and prevent harm from AI mistakes.</p>

<h3 style=3D"font-size:110%;font-weight:bold">Human Mistakes vs AI Mistake= s</h3>

<p>Life experience makes it fairly easy for each of us to guess when and w= here humans will make mistakes. Human errors tend to come at the edges of=
someone=E2=80=99s knowledge: Most of us would make mistakes solving calcu=
lus problems. We expect human mistakes to be clustered: A single calculus=
mistake is likely to be accompanied by others. We expect mistakes to wax=
and wane=2C predictably depending on factors such as fatigue and distract= ion. And mistakes are often accompanied by ignorance: Someone who makes ca= lculus mistakes is also likely to respond =E2=80=9CI don=E2=80=99t know=E2= =80=9D to calculus-related questions.</p>

<p>To the extent that AI systems make these human-like mistakes=2C we can=
bring all of our mistake-correcting systems to bear on their output. But=
the current crop of AI models -- particularly LLMs -- make mistakes diffe= rently.</p>

<p>AI errors come at seemingly random times=2C without any clustering arou=
nd particular topics. LLM mistakes tend to be more evenly distributed thro=
ugh the knowledge space. A model might be equally likely to make a mistake=
on a calculus question as it is to propose that <a href=3D"https://arxiv.= org/html/2405.19616v1">cabbages</a> eat goats.</p>

<p>And AI mistakes aren=E2=80=99t accompanied by ignorance. A LLM will be=
<a href=3D"https://spectrum.ieee.org/chatgpt-reliability">just as confide= nt</a> when saying something completely wrong -- and obviously so=2C to a=
human -- as it will be when saying something true. The seemingly random <=
a href=3D"https://arxiv.org/pdf/2305.14279">inconsistency</a> of LLMs make=
s it hard to trust their reasoning in complex=2C multi-step problems. If y=
ou want to use an AI model to help with a business problem=2C it=E2=80=99s=
not enough to see that it understands what factors make a product profita= ble; you need to be sure it won=E2=80=99t forget what money is.</p>

<h3 style=3D"font-size:110%;font-weight:bold">How to Deal with AI Mistakes= </h3>

<p>This situation indicates two possible areas of research. The first is t=
o engineer LLMs that make more human-like mistakes. The second is to build=
new mistake-correcting systems that deal with the specific sorts of mista=
kes that LLMs tend to make.</p>

<p>We already have some tools to lead LLMs to act in more human-like ways.=
Many of these arise from the field of =E2=80=9C<a href=3D"https://arxiv.o= rg/abs/2406.18346">alignment</a>=E2=80=9D research=2C which aims to make m= odels <a href=3D"https://spectrum.ieee.org/the-alignment-problem-openai">a=
ct in accordance</a> with the goals and motivations of their human develop= ers. One example is the technique that was <a href=3D"https://venturebeat.= com/ai/how-reinforcement-learning-with-human-feedback-is-unlocking-the-pow= er-of-generative-ai/">arguably</a> responsible for the breakthrough succes=
s of <a href=3D"https://spectrum.ieee.org/tag/chatgpt">ChatGPT</a>: <a hre= f=3D"https://arxiv.org/abs/2203.02155">reinforcement learning with human f= eedback</a>. In this method=2C an AI model is (figuratively) rewarded for=
producing responses that get a thumbs-up from human evaluators. Similar a= pproaches could be used to induce AI systems to make more human-like mista= kes=2C particularly by penalizing them more for mistakes that are less int= elligible.</p>

<p>When it comes to catching AI mistakes=2C some of the systems that we us=
e to prevent human mistakes will help. To an extent=2C forcing LLMs to <a=
href=3D"https://arxiv.org/pdf/2308.00436">double-check</a> their own work=
can help prevent errors. But LLMs can also <a href=3D"https://arxiv.org/p= df/2406.02061">confabulate</a> seemingly plausible=2C but truly ridiculous=
=2C explanations for their flights from reason.</p>

<p>Other mistake mitigation systems for AI are unlike anything we use for=
humans. Because machines can=E2=80=99t get fatigued or frustrated in the=
way that humans do=2C it can help to ask an LLM the same question repeate=
dly in slightly different ways and then <a href=3D"https://arxiv.org/abs/2= 210.02441">synthesize</a> its multiple responses. Humans won=E2=80=99t put=
up with that kind of annoying repetition=2C but machines will.</p>

<h3 style=3D"font-size:110%;font-weight:bold">Understanding Similarities a=
nd Differences</h3>

<p>Researchers are still struggling to understand where LLM mistakes diver=
ge from human ones. Some of the weirdness of AI is actually more human-lik=
e than it first appears. Small changes to a query to an LLM can result in=
wildly different responses=2C a problem known as <a href=3D"https://arxiv= =2Eorg/pdf/2311.07230">prompt sensitivity</a>. But=2C as any survey research= er can tell you=2C humans behave this way=2C too. The phrasing of a questi=
on in an opinion poll can have drastic <a href=3D"https://psycnet.apa.org/= record/1992-97329-001">impacts</a> on the answers.</p>

<p>LLMs also seem to have a bias towards <a href=3D"http://proceedings.mlr= =2Epress/v139/zhao21c/zhao21c.pdf">repeating</a> the words that were most co= mmon in their training data; for example=2C guessing familiar place names=
like =E2=80=9CAmerica=E2=80=9D even when asked about more exotic location=
s. Perhaps this is an example of the human =E2=80=9C<a href=3D"https://arx= iv.org/pdf/2305.04400">availability heuristic</a>=E2=80=9D manifesting in=
LLMs=2C with machines spitting out the first thing that comes to mind rat=
her than reasoning through the question. And like humans=2C perhaps=2C som=
e LLMs seem to get <a href=3D"https://arxiv.org/html/2404.08865v1">distrac= ted</a> in the middle of long documents; they=E2=80=99re better able to re= member facts from the beginning and end. There is already progress on impr= oving this error mode=2C as researchers have found that LLMs trained on <a=
href=3D"https://www.anthropic.com/news/claude-2-1-prompting">more example= s</a> of retrieving information from long texts seem to do better at retri= eving information uniformly.</p>

<p>In some cases=2C what=E2=80=99s bizarre about LLMs is that they act mor=
e like humans than we think they should. For example=2C some researchers h=
ave tested the <a href=3D"https://minimaxir.com/2024/02/chatgpt-tips-analy= sis/">hypothesis</a> that LLMs perform better when offered a cash reward o=
r threatened with death. It also turns out that some of the best ways to=
=E2=80=9C<a href=3D"https://www.usenix.org/system/files/sec24fall-prepub-= 1500-yu-zhiyuan.pdf">jailbreak</a>=E2=80=9D LLMs (getting them to disobey=
their creators=E2=80=99 explicit instructions) look a lot like the kinds=
of social engineering tricks that humans use on each other: for example=
=2C pretending to be someone else or saying that the request is just a jok=
e. But other effective jailbreaking techniques are things no human would e=
ver fall for. One group <a href=3D"https://arxiv.org/abs/2402.11753">found=
</a> that if they used <a href=3D"https://en.wikipedia.org/wiki/ASCII_art"=

ASCII art</a> (constructions of symbols that look like words or pictures)=

to pose dangerous questions=2C like how to build a bomb=2C the LLM would=
answer them willingly.</p>

<p>Humans may occasionally make seemingly random=2C incomprehensible=2C an=
d inconsistent mistakes=2C but such occurrences are rare and often indicat=
ive of more serious problems. We also tend not to put people exhibiting th=
ese behaviors in decision-making positions. Likewise=2C we should confine=
AI decision-making systems to applications that suit their actual abiliti=
es -- while keeping the potential ramifications of their mistakes firmly i=
n mind.</p>

<p><em>This essay was written with Nathan E. Sanders=2C and originally app= eared in <a href=3D"https://spectrum.ieee.org/ai-mistakes-schneier">IEEE S= pectrum</a>.</em></p>

<p>EDITED TO ADD (1/24): Slashdot <a href=3D"https://slashdot.org/story/25= /01/23/1645242/ai-mistakes-are-very-different-from-human-mistakes">thread<= /a>.</p>

<p style=3D"font-size:88%">** *** ***** ******* *********** *************<=


<h2 style=3D"font-size:125%;font-weight:bold" id=3D"cg6"><a name=3D"cg6">A=
I Will Write Complex Laws</a></h2>

<p><a href=3D"https://www.schneier.com/blog/archives/2025/01/ai-will-write= -complex-laws.html"><strong>[2025.01.22]</strong></a> Artificial intellig=
ence (AI) is writing law today. This has required no changes in legislativ=
e procedure or the rules of legislative bodies -- all it takes is one legi= slator=2C or legislative assistant=2C to use generative AI in the process=
of drafting a bill.</p>

<p>In fact=2C the use of AI by legislators is only likely to become more p= revalent. There are currently projects in the US House=2C US Senate=2C and=
<a href=3D"https://www.popvox.org/blog/assessing-us-congressional-ai-adop= tion">legislatures around the world</a> to trial the use of AI in various=
ways: searching databases=2C drafting text=2C summarizing meetings=2C per= forming policy research and analysis=2C and more. A Brazilian municipality=
<a href=3D"https://apnews.com/article/brazil-artificial-intelligence-port= o-alegre-5afd1240afe7b6ac202bb0bbc45e08d4">passed the first known AI-writt=
en law</a> in 2023.</p>

<p>That=E2=80=99s not surprising; AI is being used more everywhere. What i=
s coming into focus is how policymakers will use AI and=2C critically=2C h=
ow this use will change the balance of power between the legislative and e= xecutive branches of government. Soon=2C US legislators may turn to AI to=
help them keep pace with the increasing complexity of their lawmaking --=
and this will suppress the power and discretion of the executive branch t=
o make policy.</p>

<h3 style=3D"font-size:110%;font-weight:bold">Demand for Increasingly Comp=
lex Legislation</h3>

<p>Legislators are writing increasingly long=2C intricate=2C and complicat=
ed laws that human legislative drafters have trouble producing. Already in=
the US=2C the <a href=3D"https://www.cambridge.org/core/journals/american= -political-science-review/article/abs/lobbying-as-legislative-subsidy/AE4B= 5D8AB9C2487BB78C2A51BB53E03F">multibillion-dollar lobbying industry is sub= sidizing lawmakers</a> in writing baroque laws: suggesting paragraphs to a=
dd to bills=2C specifying benefits for some=2C carving out exceptions for=
others. Indeed=2C the <a href=3D"https://www.oecd.org/en/publications/lob= bying-in-the-21st-century_c6d8eff8-en.html">lobbying industry</a> is growi=
ng in complexity and influence worldwide.</p>

<p>Several years ago=2C <a href=3D"http://dx.doi.org/10.1177/1532673X18776= 628">researchers studied bills</a> introduced into state legislatures thro= ughout the US=2C looking at which bills were wholly original texts and whi=
ch borrowed text from other states or from lobbyist-written model legislat= ion. Their conclusion was not very surprising. Those who borrowed the most=
text were in legislatures that were less resourced. This makes sense: If=
you=E2=80=99re a part-time legislator=2C perhaps unpaid and without a lot=
of staff=2C you need to rely on more external support to draft legislatio=
n. When the scope of policymaking outstrips the resources of legislators=
=2C they look for help. Today=2C that often means lobbyists=2C who provide=
expertise=2C research services=2C and drafting labor to legislators at th=
e local=2C state=2C and federal levels at no charge. Of course=2C they are=
not unbiased: They seek to exert influence on behalf of their clients.</p=


<p><a href=3D"https://doi.org/10.1086/714933">Ano</a><a href=3D"https://do= i.org/10.1086/714933">ther study</a>=2C at the US federal level=2C measure=
d the complexity of policies proposed in legislation and tried to determin=
e the factors that led to such growing complexity. While there are <a href= =3D"https://core.ac.uk/download/pdf/217430499.pdf">numerous ways</a> to me= asure legal complexity=2C these authors focused on the specificity of inst= itutional design: How exacting is Congress in laying out the relational ne= twork of branches=2C agencies=2C and officials that will share power to im= plement the policy?</p>

<p>In looking at bills enacted between 1993 and 2014=2C the researchers fo=
und two things. First=2C they concluded that ideological polarization driv=
es complexity. The suggestion is that if a legislator is on the extreme en=
d of the ideological spectrum=2C they=E2=80=99re more likely to introduce=
a complex law that constrains the discretion of=2C as the authors put it=
=2C =E2=80=9Centrenched bureaucratic interests.=E2=80=9D And second=2C the=
y found that divided government drives complexity to a large degree: Signi= ficant legislation passed under divided government was found to be 65 perc=
ent more complex than similar legislation passed under unified government.=
Their conclusion is that=2C if a legislator=E2=80=99s party controls Cong= ress=2C and the opposing party controls the White House=2C the legislator=
will want to give the executive as little wiggle room as possible. When l= egislators=E2=80=99 preferences disagree with the executive=E2=80=99s=2C t=
he legislature is incentivized to write laws that specify all the details.=
This gives the agency designated to implement the law as little discretio=
n as possible.</p>

<p>Because polarization and divided government are increasingly entrenched=
in the US=2C the demand for complex legislation at the federal level is l= ikely to grow. Today=2C we have both the <a href=3D"https://www.pewresearc= h.org/short-reads/2022/03/10/the-polarization-in-todays-congress-has-roots= -that-go-back-decades">greatest ideological polarization</a> in Congress i=
n living memory and an increasingly divided government at the federal leve=
l. Between 1900 and 1970 (57th through 90th Congresses)=2C we had 27 insta= nces of unified government and only seven divided; nearly a four-to-one ra= tio. Since then=2C the trend is roughly the opposite. As of the start of t=
he next Congress=2C we will have had 20 divided governments and only eight=
unified (nearly a three-to-one ratio). And while the incoming Trump admin= istration will see a unified government=2C the extremely closely divided H= ouse may often make this Congress look and feel like a divided one (see th=
e <a href=3D"https://www.washingtonpost.com/politics/2024/12/21/trump-musk= -shutdown-demands-defeat/">recent government shutdown crisis</a> as an exe= mplar) and makes truly divided government a strong possibility in 2027.</p=


<p>Another related factor driving the complexity of legislation is the nee=
d to do it all at once. The lobbyist <a href=3D"https://onlinelibrary.wile= y.com/doi/full/10.1111/lsq.12266">feeding frenzy</a> -- spurring major bil=
ls like the Affordable Care Act to be thousands of pages in length -- is d= riven in part by gridlock in Congress. <a href=3D"https://abcnews.go.com/P= olitics/118th-congress-track-become-productive-us-history/story?id=3D10625= 4012">Congressional productivity has dropped so low</a> that bills on any=
given policy issue seem like a once-in-a-generation opportunity for legis= lators -- and lobbyists -- to set policy.</p>

<p>These dynamics also impact the states. States often have divided govern= ments=2C <a href=3D"https://www.multistate.us/insider/2024/2/13/state-trif= ectas-hit-a-new-record-in-2024-only-10-states-have-divided-government">alb=
eit less often than they used to</a>=2C and their demand for drafting assi= stance is arguably higher due to their significantly smaller staffs. And s= ince the <a href=3D"https://www.reuters.com/graphics/USA-CONGRESS/PRODUCTI= VITY/egpbabmkwvq/">productivity of Congress has cratered</a> in recent yea= rs=2C significantly more policymaking is happening at the state level.</p>

<p>But there=E2=80=99s another reason=2C particular to the US federal gove= rnment=2C that will likely force congressional legislation to be more comp=
lex even during unified government. In June 2024=2C the <a href=3D"https:/= /www.scotusblog.com/2024/06/supreme-court-strikes-down-chevron-curtailing-= power-of-federal-agencies">US Supreme Court overturned the </a><a href=3D"= https://www.scotusblog.com/2024/06/supreme-court-strikes-down-chevron-curt= ailing-power-of-federal-agencies"><em>Chevron</em></a><a href=3D"https://w= ww.scotusblog.com/2024/06/supreme-court-strikes-down-chevron-curtailing-po= wer-of-federal-agencies"> doctrine</a>=2C which gave executive agencies br=
oad power to specify and implement legislation. Suddenly=2C there is <a hr= ef=3D"https://www.klgates.com/What-Overturning-Chevron-Means-for-the-Way-C= ongress-Does-Its-Business-7-18-2024">a mandate from the Supreme Court</a>=
for more specific legislation. Issues that have historically been left im= plicitly to the executive branch are now required to be either explicitly=
delegated to agencies or specified directly in statute. Either way=2C the=
Court=E2=80=99s ruling implied that law should become more complex and th=
at Congress should increase its <a href=3D"https://bipartisanpolicy.org/bl= og/building-congress-for-post-chevron-world/">policymaking capacity</a>.</=


<p>This affects the balance of power between the executive and legislative=
branches of government. When the legislature delegates less to the execut=
ive branch=2C it increases its own power. Every decision made explicitly i=
n statute is a decision the executive makes not on its own but=2C rather=
=2C according to the directive of the legislature. In the US system of <a=
href=3D"https://core.ac.uk/download/pdf/80562076.pdf">separation of power= s</a>=2C administrative law is a tool for balancing power among the legisl= ative=2C executive=2C and judicial branches. The legislature gets to decid=
e when to delegate and when not to=2C and it can respond to judicial revie=
w to adjust its delegation of control as needed. The elimination of <em>Ch= evron</em> will induce the legislature to exert its control over delegatio=
n more robustly.</p>

<p>At the same time=2C there are powerful political <a href=3D"https://goo= dauthority.org/news/abolishing-chevron-could-undermine-congress-scotus-lop= er">incentives for Congress to be vague</a> and to rely on someone else=2C=
like agency bureaucrats=2C to make hard decisions. That empowers third pa= rties -- the corporations=2C or lobbyists -- that have been gifted by the=
overturning of <em>Chevron</em> a new tool in arguing against administrat=
ive regulations not specifically backed up by law. A <a href=3D"https://ww= w.nytimes.com/2024/06/28/us/politics/supreme-court-regulatory-agencies.htm= l">continuing stream of Supreme Court decisions</a> handing victories to u= npopular industries could be another driver of complex law=2C adding polit= ical pressure to pass legislative fixes.</p>

<h3 style=3D"font-size:110%;font-weight:bold">AI Can Supply Complex Legisl= ation</h3>

<p>Congress may or may not be up to the challenge of putting more policy d= etails into law=2C but the external forces outlined above -- lobbyists=2C=
the judiciary=2C and an increasingly divided and polarized government --=
are pushing them to do so. When Congress does take on the task of writing=
complex legislation=2C it=E2=80=99s quite likely it will turn to AI for h= elp.</p>

<p>Two particular AI capabilities enable Congress to write laws different=
from laws humans tend to write. One=2C AI models have an enormous <em>sco= pe</em> of expertise=2C whereas people have only a handful of specializati= ons. Large language models (LLMs) like the one powering ChatGPT can genera=
te legislative text on funding specialty crop harvesting mechanization equ= ally as well as material on energy efficiency standards for street lightin=
g. This enables a legislator to address more topics simultaneously. Two=2C=
AI models have the <em>sophistication</em> to work with a higher degree o=
f complexity than people can. Modern LLM systems can instantaneously perfo=
rm <a href=3D"https://arxiv.org/pdf/2411.05000">several simultaneous multi= step reasoning tasks</a> using information from thousands of pages of docu= ments. This enables a legislator to fill in more baroque detail on any giv=
en topic.</p>

<p>That=E2=80=99s not to say that handing over legislative drafting to mac= hines is easily done. Modernizing <a href=3D"https://www.recodingamerica.u= s">any institutional process is extremely hard</a>=2C even when the techno= logy is readily available and performant. And <a href=3D"https://proceedin= gs.neurips.cc/paper_files/paper/2023/file/89e44582fd28ddfea1ea4dcb0ebbf4b0= -Paper-Datasets_and_Benchmarks.pdf">modern AI still has a ways to go</a> t=
o achieve mastery of complex legal and policy issues. But the basic tools=
are there.</p>

<p>AI can be used in each step of lawmaking=2C and this will bring various=
benefits to policymakers. It could let them work on more policies -- more=
bills -- at the same time=2C add more detail and specificity to each bill=
=2C or interpret and <a href=3D"https://static.ie.edu/CGC/AI4D%20Paper%203= %20Applications%20of%20Artificial%20Intelligence%20Tools%20to%20Engance%20= Legislative%20Engagement.pdf">incorporate more feedback from constituents<=

and outside groups. The addition of a single AI tool to a legislative=

office may have an impact similar to adding several people to their staff=
=2C but with far lower cost.</p>

<p>Speed sometimes matters when writing law. When there is a change of gov= erning party=2C there is often a rush to change as much policy as possible=
to match the platform of the new regime. AI could help legislators do tha=
t kind of wholesale revision. The result could be policy that is more resp= onsive to voters -- or more political instability. Already in 2024=2C the=
US House=E2=80=99s <a href=3D"https://cha.house.gov/_cache/files/7/5/75fd= a5da-b1e4-4990-a543-6ca495f983ef/CD88E4BBA4CE5ABDD53A31C31931168B.cha-mode= rnization-ai-flash-report-07-30-24-51-.pdf">Office of the Clerk has begun=
using AI</a> to speed up the process of producing cost estimates for bill=
s and understanding how new legislation relates to existing code. <a href= =3D"https://search-prod.lis.state.oh.us/api/v2/general_assembly_135/commit= tees/cmte_s_govt_1/meetings/cmte_s_govt_1_2024-05-22-1030_1242/submissions= /olr_testimony_csi.pdf">Ohio has used an AI tool</a> to do wholesale revis=
ion of state administrative law since 2020.</p>

<p>AI can also make laws clearer and more consistent. With their superhuma=
n attention spans=2C AI tools are good at enforcing syntactic and grammati=
cal rules. They will be effective at drafting text in precise and proper l= egislative language=2C or offering detailed feedback to human drafters. Bo= rrowing ideas from software development=2C where coders use tools to ident=
ify common instances of bad programming practices=2C an AI reviewer can <a=
href=3D"https://link.springer.com/article/10.1007/s10506-022-09315-w">hig= hlight bad law-writing practices</a>. For example=2C it can detect when si= gnificant phrasing is inconsistent across a long bill. If a bill about ins= urance repeatedly lists a variety of disaster categories=2C but leaves one=
out one time=2C AI can catch that.</p>

<p>Perhaps this seems like minutiae=2C but a small ambiguity or mistake in=
law can have massive consequences. In 2015=2C the Affordable Care Act cam=
e close to being struck down <a href=3D"https://www.nytimes.com/2015/05/26= /us/politics/contested-words-in-affordable-care-act-may-have-been-left-by-= mistake.html">because of a typo in four words</a>=2C imperiling health car=
e services extended to more than 7 million Americans.</p>

<p>There=E2=80=99s more that AI can do in the legislative process. AI can=
summarize bills and answer questions about their provisions. It can highl= ight aspects of a bill that align with=2C or are contrary to=2C different=
political points of view. We can even imagine a future in which AI can be=
used to simulate a new law and determine whether or not it would be effec= tive=2C or what the side effects would be. This means that beyond writing=
them=2C AI could help lawmakers <em>understand</em> laws. Congress is not= orious for producing bills hundreds of pages long=2C and many other countr=
ies sometimes have similarly massive omnibus bills that address many issue=
s at once. It=E2=80=99s impossible for any one person to understand how ea=
ch of these bills=E2=80=99 provisions would work. Many legislatures employ=
human analysis in budget or fiscal offices that analyze these bills and o= ffer reports. AI could do this kind of work at greater speed and scale=2C=
so legislators could easily query an AI tool about how a particular bill=
would affect their district or areas of concern.</p>

<p>This is a use case that the <a href=3D"https://cha.house.gov/modernizat= ion">House </a><a href=3D"https://cha.house.gov/modernization">subcommitte= e</a> on modernization <a href=3D"https://cha.house.gov/_cache/files/0/8/0= 8476380-95c3-4989-ad4c-1e2a454b0007/9668ADB6A0D503B944E26EDB81EDC585.cha-m= odernization-ai-flash-report-10-25-24.pdf">has urged the Library of Congre= ss</a> to take action on. Numerous software vendors are <a href=3D"https:/= /www.law.com/legaltechnews/2024/11/22/legaltech-rundown-lexisnexis-release= s-lexis-ai-mobile-app-hotshot-launches-new-ma-training-simulation-and-more= ">already marketing AI legislative</a> analysis tools. These tools can pot= entially find loopholes or=2C <a href=3D"https://www.technologyreview.com/= 2023/03/14/1069717/how-ai-could-write-our-laws/">like the human lobbyists=
of today</a>=2C craft them to benefit particular private interests.</p>

<p>These capabilities will be attractive to legislators who are looking to=
expand their power and capabilities but don=E2=80=99t necessarily have mo=
re funding to hire human staff. We should understand the idea of AI-augmen=
ted lawmaking contextualized within the longer history of legislative tech= nologies. To serve society at modern scales=2C we=E2=80=99ve had to come a=
long way from the Athenian ideals of direct democracy and sortition. Demo= cracy no longer involves just one person and one vote to decide a policy.=
It involves hundreds of thousands of constituents electing one representa= tive=2C who is augmented by a staff as well as subsidized by lobbyists=2C=
and who implements policy through a vast administrative state coordinated=
by digital technologies. Using AI to help those representatives specify a=
nd refine their policy ideas is part of a long history of transformation.<=


<p>Whether all this AI augmentation is good for all of us subject to the l=
aws they make is less clear. There are real risks to AI-written law=2C but=
those risks are not dramatically different from what we endure today. AI-= written law trying to optimize for certain policy outcomes may get it wron=
g (just as many human-written laws are misguided). AI-written law may be m= anipulated to benefit one constituency over others=2C by the tech companie=
s that develop the AI=2C or by the legislators who apply it=2C just as hum=
an lobbyists steer policy to benefit their clients.</p>

<p>Regardless of what anyone thinks of any of this=2C regardless of whethe=
r it will be a net positive or a net negative=2C AI-made legislation is co= ming -- the growing complexity of policy demands it. It doesn=E2=80=99t re= quire any changes in legislative procedures or agreement from any rules co= mmittee. All it takes is for one legislative assistant=2C or lobbyist=2C t=
o fire up a chatbot and ask it to create a draft. When legislators voted o=
n that Brazilian bill in 2023=2C <a href=3D"https://apnews.com/article/bra= zil-artificial-intelligence-porto-alegre-5afd1240afe7b6ac202bb0bbc45e08d4"= >they didn=E2=80=99t know it was AI-written</a>; the use of ChatGPT was un= disclosed. And even if they had known=2C it=E2=80=99s not clear it would h=
ave made a difference. In the future=2C as in the past=2C we won=E2=80=99t=
always know which laws will have good impacts and which will have bad eff= ects=2C regardless of the words on the page=2C or who (or what) wrote them= =2E</p>

<p><em>This essay was written with Nathan E. Sanders=2C and originally app= eared in <a href=3D"https://www.lawfaremedia.org/article/ai-will-write-com= plex-laws">Lawfare</a>.</em></p>

<p style=3D"font-size:88%">** *** ***** ******* *********** *************<=


<h2 style=3D"font-size:125%;font-weight:bold" id=3D"cg7"><a name=3D"cg7">T= hird Interdisciplinary Workshop on Reimagining Democracy (IWORD 2024)</a><=


<p><a href=3D"https://www.schneier.com/blog/archives/2025/01/third-interdi= sciplinary-workshop-on-reimagining-democracy-iword-2024.html"><strong>[20= 25.01.23]</strong></a> Last month=2C Henry Farrell and I convened the Thir=
d Interdisciplinary Workshop on Reimagining Democracy (<a href=3D"https://= www.schneier.com/iword/2024/">IWORD 2024</a>) at Johns Hopkins University= =E2=80=99s Bloomberg Center in Washington DC. This is a small=2C invitatio=
nal workshop on the future of democracy. As with the <a href=3D"https://ww= w.schneier.com/iword/2022/">previous</a> <a href=3D"https://www.schneier.c= om/iword/2023/">two</a> workshops=2C the goal was to bring together a dive=
rse set of political scientists=2C law professors=2C philosophers=2C AI re= searchers and other industry practitioners=2C political activists=2C and c= reative types (including science fiction writers) to discuss how democracy=
might be reimagined in the current century.</p>

<p>The goal of the workshop is to think very broadly. Modern democracy was=
invented in the mid-eighteenth century=2C using mid-eighteenth-century te= chnology. If democracy were to be invented today=2C it would look very dif= ferent. Elections would look different. The balance between representation=
and direct democracy would look different. Adjudication and enforcement w= ould look different. Everything would look different=2C because our concep= tions of fairness=2C justice=2C equality=2C and rights are different=2C an=
d we have much more powerful technology to bring to bear on the problems.=
Also=2C we could start from scratch without having to worry about evolvin=
g our current democracy into this imagined future system.</p>

<p>We can=E2=80=99t do that=2C of course=2C but it=E2=80=99s still still v= aluable to speculate. Of course we need to figure out how to reform our cu= rrent systems=2C but we shouldn=E2=80=99t limit our thinking to incrementa=
l steps. We also need to think about discontinuous changes as well. I wrot=
e about the philosophy more in this <a href=3D"https://theconversation.com= /re-imagining-democracy-for-the-21st-century-possibly-without-the-trapping= s-of-the-18th-century-210586">essay</a> about IWORD 2022.</p>

<p>IWORD 2024 was easily the most intellectually stimulating two days of m=
y year. It=E2=80=99s also intellectually exhausting; the speed and intensi=
ty of ideas is almost too much. I wrote about the format in my <a href=3D"= https://www.schneier.com/blog/archives/2024/01/second-interdisciplinary-wo= rkshop-on-reimagining-democracy.html">blog post</a> on IWORD 2023.</p>

<p>Summaries of all the IWORD 2024 talks are in the first set of comments=
below. And here are links to the previous IWORDs:</p>



<li>IWORD 2022: <a href=3D"https://www.schneier.com/iword/2022/">home=
page</a>=2C <a href=3D"https://theconversation.com/re-imagining-democracy= -for-the-21st-century-possibly-without-the-trappings-of-the-18th-century-2= 10586">essay</a>=2C and <a href=3D"https://www.schneier.com/blog/archives/= 2022/12/reimagining-democracy.html">talk summaries</a></li>

<li>IWORD 2023: <a href=3D"https://www.schneier.com/iword/2023/">home=
page</a> and <a href=3D"https://www.schneier.com/blog/archives/2024/01/se= cond-interdisciplinary-workshop-on-reimagining-democracy.html">talk summar= ies</a>.</li>
</ul>

<p>IWORD 2025 will be held either in New York or New Haven; still to be de= termined.</p>

<p style=3D"font-size:88%">** *** ***** ******* *********** *************<=


<h2 style=3D"font-size:125%;font-weight:bold" id=3D"cg8"><a name=3D"cg8">N=
ew VPN Backdoor</a></h2>

<p><a href=3D"https://www.schneier.com/blog/archives/2025/01/new-vpn-backd= oor.html"><strong>[2025.01.27]</strong></a> A newly discovered <a href=3D= "https://arstechnica.com/security/2025/01/backdoor-infecting-vpns-used-mag= ic-packets-for-stealth-and-security/">VPN backdoor</a> uses some interesti=
ng tactics to avoid detection:</p>

<blockquote><p>When threat actors use backdoor malware to gain access to a=
network=2C they want to make sure all their hard work can=E2=80=99t be le= veraged by competing groups or detected by defenders. One countermeasure i=
s to equip the backdoor with a passive agent that remains dormant until it=
receives what=E2=80=99s known in the business as a =E2=80=9Cmagic packet.= =E2=80=9D On Thursday=2C researchers revealed that a never-before-seen bac= kdoor that quietly took hold of dozens of enterprise VPNs running Juniper=
Network=E2=80=99s Junos OS has been doing just that.</p>

<p>J-Magic=2C the tracking name for the backdoor=2C goes one step further=
to prevent unauthorized access. After receiving a magic packet hidden in=
the normal flow of TCP traffic=2C it relays a challenge to the device tha=
t sent it. The challenge comes in the form of a string of text that=E2=80=
=99s encrypted using the public portion of an RSA key. The initiating part=
y must then respond with the corresponding plaintext=2C proving it has acc=
ess to the secret key.</p>

<p>The lightweight backdoor is also notable because it resided only in mem= ory=2C a trait that makes detection harder for defenders. The combination=
prompted researchers at Lumin Technology=E2=80=99s Black Lotus Lab to sit=
up and take notice.</p>

<p>[...]</p>

<p>The researchers found J-Magic on <a href=3D"https://www.virustotal.com/= gui/home/upload">VirusTotal</a> and determined that it had run inside the=
networks of 36 organizations. They still don=E2=80=99t know how the backd=
oor got installed.</p></blockquote>

<p>Slashdot <a href=3D"https://tech.slashdot.org/story/25/01/24/0039249/ba= ckdoor-infecting-vpns-used-magic-packets-for-stealth-and-security">thread<= /a>.</p>

<p>EDITED TO ADD (2/1): Another <a href=3D"https://www.theregister.com/202= 5/01/25/mysterious_backdoor_juniper_routers/">article</a>.</p>

<p style=3D"font-size:88%">** *** ***** ******* *********** *************<=


<h2 style=3D"font-size:125%;font-weight:bold" id=3D"cg9"><a name=3D"cg9">C=
ISA Under Trump</a></h2>

<p><a href=3D"https://www.schneier.com/blog/archives/2025/01/cisa-under-tr= ump.html"><strong>[2025.01.28]</strong></a> Jen Easterly is out as the Di= rector of CISA. Read her final <a href=3D"https://www.wired.com/story/big-= interview-jen-easterly-cisa-cybersecurity/">interview</a>:</p>

<blockquote><p>There=E2=80=99s a lot of unfinished business. We have made=
an impact through our ransomware vulnerability warning pilot and our pre-= ransomware notification initiative=2C and I=E2=80=99m really proud of that=
=2C because we work on preventing somebody from having their worst day. Bu=
t ransomware is still a problem. We have been laser-focused on PRC cyber a= ctors. That will continue to be a huge problem. I=E2=80=99m really proud o=
f where we are=2C but there=E2=80=99s much=2C much more work to be done. T= here are things that I think we can continue driving=2C that the next admi= nistration=2C I hope=2C will look at=2C because=2C frankly=2C cybersecurit=
y is a national security issue.</p></blockquote>

<p>If Project 2025 is a guide=2C the agency will be <a href=3D"https://www= =2Edemocracydocket.com/analysis/a-little-known-federal-agency-helps-secure-e= lections-trump-wants-to-gut-it/">gutted</a> under Trump:</p>

<blockquote><p>=E2=80=9CProject 2025=E2=80=99s recommendations -- essentia=
lly because this one thing caused anger -- is to just strip the agency of=
all of its support altogether=2C=E2=80=9D he said. =E2=80=9CAnd CISA=E2= =80=99s functions go so far beyond its role in the information space in a=
way that would do real harm to election officials and leave them less pre= pared to tackle future challenges.=E2=80=9D</p>

<p>In the DHS chapter of Project 2025=2C Cucinelli suggests gutting CISA a= lmost entirely=2C moving its core responsibilities on critical infrastruct=
ure to the Department of Transportation. It=E2=80=99s a suggestion that Ad=
av Noti=2C the executive director of the nonpartisan voting rights advocac=
y organization Campaign Legal Center=2C <a href=3D"https://www.democracydo= cket.com/analysis/unmasking-the-anti-democracy-agenda-of-project-2025/">pr= eviously described</a> to Democracy Docket as =E2=80=9Cabsolutely bonkers.= =E2=80=9D</p>

<p>=E2=80=9CIt=E2=80=99s located at Homeland Security because the whole pr= emise of the Department of Homeland Security is that it=E2=80=99s supposed=
to be the central resource for the protection of the nation=2C=E2=80=9D N=
oti said. =E2=80=9CAnd that the important functions shouldn=E2=80=99t be l= iving out in siloed agencies.=E2=80=9D</p></blockquote>

<p style=3D"font-size:88%">** *** ***** ******* *********** *************<=


<h2 style=3D"font-size:125%;font-weight:bold" id=3D"cg10"><a name=3D"cg10"= >ExxonMobil Lobbyist Caught Hacking Climate Activists</a></h2>

<p><a href=3D"https://www.schneier.com/blog/archives/2025/01/exxonmobil-lo= bbyist-caught-hacking-climate-activists.html"><strong>[2025.01.29]</stron= g></a> The Department of Justice is investigating a lobbying firm represen= ting ExxonMobil for <a href=3D"https://www.npr.org/2025/01/24/nx-s1-527153= 0/hacking-investigation-climate-change">hacking</a> the phones of climate=
activists:</p>

<blockquote><p>The hacking was allegedly commissioned by a Washington=2C D= =2EC.=2C lobbying firm=2C <a href=3D"https://legacy.www.documentcloud.org/do= cuments/25501845-250113-usa-v-forlit/">according to a lawyer representing=
the U.S. government</a>. The firm=2C in turn=2C was allegedly working on=
behalf of one of the world=E2=80=99s largest oil and gas companies=2C bas=
ed in Texas=2C that wanted to discredit groups and individuals involved in=
climate litigation=2C according to the lawyer for the U.S. government. In=
court documents=2C the Justice Department does not name either company.</=


<p>As part of its probe=2C the U.S. is trying to extradite an Israeli priv=
ate investigator named Amit Forlit from the United Kingdom for allegedly o= rchestrating the hacking campaign. A <a href=3D"https://legacy.www.documen= tcloud.org/documents/25501846-usa-v-amit-forlit-defence-skeleton-argument/= ">lawyer for Forlit claimed in a court filing</a> that the hacking operati=
on her client is accused of leading =E2=80=9Cis alleged to have been commi= ssioned by DCI Group=2C a lobbying firm representing ExxonMobil=2C one of=
the world=E2=80=99s largest fossil fuel companies.=E2=80=9D</p></blockquo=


<p style=3D"font-size:88%">** *** ***** ******* *********** *************<=


<h2 style=3D"font-size:125%;font-weight:bold" id=3D"cg11"><a name=3D"cg11"= >Fake Reddit and WeTransfer Sites Are Pushing Malware</a></h2>

<p><a href=3D"https://www.schneier.com/blog/archives/2025/01/fake-reddit-a= nd-wetransfer-sites-are-pushing-malware.html"><strong>[2025.01.30]</stron= g></a> There are thousands of <a href=3D"https://www.bleepingcomputer.com/= news/security/hundreds-of-fake-reddit-sites-push-lumma-stealer-malware/">f= ake</a> Reddit and WeTransfer webpages that are pushing malware. They expl=
oit people who are using search engines to search sites like Reddit.</p>

<blockquote><p>Unsuspecting victims clicking on the link are taken to a fa=
ke WeTransfer site that mimicks the interface of the popular file-sharing=
service. The =E2=80=98Download=E2=80=99 button leads to the <a href=3D"ht= tps://app.any.run/tasks/a629e4b1-433b-427e-8040-79d4aa13c245">Lumma Steale=
r payload</a> hosted on =E2=80=9Cweighcobbweo[.]top.=E2=80=9D</p></blockq= uote>

<p>Boing Boing <a href=3D"https://boingboing.net/2025/01/28/fake-reddit-pa= ges-are-serving-up-malware.html">post</a>.</p>

<p style=3D"font-size:88%">** *** ***** ******* *********** *************<=


<h2 style=3D"font-size:125%;font-weight:bold" id=3D"cg12"><a name=3D"cg12"= >Journalists and Civil Society Members Using WhatsApp Targeted by Paragon=
Spyware</a></h2>

<p><a href=3D"https://www.schneier.com/blog/archives/2025/02/journalists-a= nd-civil-society-members-using-whatsapp-targeted-by-paragon-spyware.html">= <strong>[2025.02.03]</strong></a> This is yet another story of commercial=
spyware being <a href=3D"https://www.theguardian.com/technology/2025/jan/= 31/whatsapp-israel-spyware">used against</a> journalists and civil society=
members.</p>

<blockquote><p>The journalists and other civil society members were being=
alerted of a possible breach of their devices=2C with WhatsApp telling th=
e Guardian it had =E2=80=9Chigh confidence=E2=80=9D that the 90 users in q= uestion had been targeted and =E2=80=9Cpossibly compromised.=E2=80=9D</p>

<p>It is not clear who was behind the attack. Like other spyware makers=2C=
Paragon=E2=80=99s hacking software is used by government clients and What= sApp said it had not been able to identify the clients who ordered the all= eged attacks.</p>

<p>Experts said the targeting was a =E2=80=9Czero-click=E2=80=9D attack=2C=
which means targets would not have had to click on any malicious links to=
be infected.</p></blockquote>

<p style=3D"font-size:88%">** *** ***** ******* *********** *************<=


<h2 style=3D"font-size:125%;font-weight:bold" id=3D"cg13"><a name=3D"cg13"= >Deepfakes and the 2024 US Election</a></h2>

<p><a href=3D"https://www.schneier.com/blog/archives/2025/02/deepfakes-and= -the-2024-us-election.html"><strong>[2025.02.04]</strong></a> Interesting=
<a href=3D"https://knightcolumbia.org/blog/we-looked-at-78-election-deepf= akes-political-misinformation-is-not-an-ai-problem">analysis</a>:</p>

<blockquote><p>We analyzed every instance of AI use in elections collected=
by the WIRED AI Elections Project (<a href=3D"https://www.cs.princeton.ed= u/~sayashk/political-misinformation/WIRED-data.html">source</a> for our an= alysis)=2C which tracked known uses of AI for creating political content d= uring elections taking place in 2024 worldwide. In each case=2C we identif=
ied what AI was used for and estimated the cost of creating similar conten=
t without AI.</p>

<p>We find that (1) half of AI use isn=E2=80=99t deceptive=2C (2) deceptiv=
e content produced using AI is nevertheless cheap to replicate <em>without= </em> AI=2C and (3) focusing on the demand for misinformation rather than=
the supply is a much more effective way to diagnose problems and identify=
interventions.</p></blockquote>

<p>This tracks with my analysis. People share as a form of social signalin=
g. I send you a meme/article/clipping/photo to show that we are on the sam=
e team. Whether it is true=2C or misinformation=2C or actual propaganda=2C=
is of secondary importance. Sometimes it=E2=80=99s completely irrelevant.=
This is why fact checking doesn=E2=80=99t work. This is why =E2=80=9Cchea=
p fakes=E2=80=9D -- obviously fake photos and videos -- are effective. Thi=
s is why=2C as the authors of that analysis said=2C the demand side is the=
real problem.</p>

<p style=3D"font-size:88%">** *** ***** ******* *********** *************<=


<h2 style=3D"font-size:125%;font-weight:bold" id=3D"cg14"><a name=3D"cg14"=

On Generative AI Security</a></h2>

<p><a href=3D"https://www.schneier.com/blog/archives/2025/02/on-generative= -ai-security.html"><strong>[2025.02.05]</strong></a> Microsoft=E2=80=99s=
AI Red Team just published =E2=80=9C<a href=3D"https://airedteamwhitepape= rs.blob.core.windows.net/lessonswhitepaper/MS_AIRT_Lessons_eBook.pdf">Less=
ons from Red Teaming 100 Generative AI Products</a>.=E2=80=9D Their <a hre= f=3D"https://www.microsoft.com/en-us/security/blog/2025/01/13/3-takeaways-= from-red-teaming-100-generative-ai-products/">blog post</a> lists =E2=80= =9Cthree takeaways=2C=E2=80=9D but the eight lessons in the report itself=
are more useful:</p>

<blockquote>


<li>Understand what the system can do and where it is applied.</li>

<li>You don=E2=80=99t have to compute gradients to break an AI system.= </li>

<li>AI red teaming is not safety benchmarking.</li>

<li>Automation can help cover more of the risk landscape.</li>

<li>The human element of AI red teaming is crucial.</li>

<li>Responsible AI harms are pervasive but difficult to measure.</li>

<li>LLMs amplify existing security risks and introduce new ones.</li>

<li>The work of securing AI systems will never be complete.</li>
</ol>
</blockquote>

<p style=3D"font-size:88%">** *** ***** ******* *********** *************<=


<h2 style=3D"font-size:125%;font-weight:bold" id=3D"cg15"><a name=3D"cg15"= >AIs and Robots Should Sound Robotic</a></h2>

<p><a href=3D"https://www.schneier.com/blog/archives/2025/02/ais-and-robot= s-should-sound-robotic.html"><strong>[2025.02.06]</strong></a> Most peopl=
e know that <a href=3D"https://spectrum.ieee.org/tag/robots">robots</a> no=
longer sound like tinny trash cans. They sound like <a href=3D"https://sp= ectrum.ieee.org/tag/siri">Siri</a>=2C <a href=3D"https://spectrum.ieee.org= /tag/alexa">Alexa</a>=2C and <a href=3D"https://spectrum.ieee.org/tag/gemi= ni">Gemini</a>. They sound like the voices in labyrinthine customer suppor=
t phone trees. And even those robot voices are being made obsolete by new=
<a href=3D"https://spectrum.ieee.org/chatgpt-multimodal">AI-generated voi= ces</a> that can mimic every vocal nuance and tic of human speech=2C down=
to specific regional accents. And with just a few seconds of audio=2C <a=
href=3D"https://spectrum.ieee.org/tag/ai">AI</a> can now <a href=3D"https= ://spectrum.ieee.org/digital-afterlife">clone someone=E2=80=99s specific v= oice</a>.</p>

<p>This technology will replace humans in many areas. Automated customer s= upport will save <a href=3D"https://spectrum.ieee.org/tag/money">money</a>=
by cutting staffing at <a href=3D"https://spectrum.ieee.org/tag/call-cent= ers">call centers</a>. <a href=3D"https://spectrum.ieee.org/ai-agents">AI=
agents</a> will make calls on our behalf=2C conversing with others in <a=
href=3D"https://spectrum.ieee.org/tag/natural-language">natural language<= /a>. All of that is happening=2C and will be commonplace soon.</p>

<p>But there is something fundamentally different about talking with a bot=
as opposed to a person. A person can be a friend. An AI cannot be a frien= d=2C despite how people might treat it or react to it. AI is at best a too= l=2C and at worst a means of manipulation. Humans need to know whether we= =E2=80=99re talking with a living=2C breathing person or a robot with an a= genda set by the person who controls it. That=E2=80=99s why robots should=
sound like robots.</p>

<p>You can=E2=80=99t just label AI-generated speech. It will come in many=
different forms. So we need a way to recognize AI that works no matter th=
e modality. It needs to work for long or short snippets of audio=2C even j=
ust a second long. It needs to work for any language=2C and in any cultura=
l context. At the same time=2C we shouldn=E2=80=99t constrain the underlyi=
ng system=E2=80=99s sophistication or language complexity.</p>

<p>We have a simple proposal: all talking AIs and robots should use a ring=
<a href=3D"https://spectrum.ieee.org/tag/modulator">modulator</a>. In the=
mid-twentieth century=2C before it was easy to create actual robotic-soun= ding speech synthetically=2C ring modulators were used to make actors=E2= =80=99 voices sound robotic. Over the last few decades=2C we have become a= ccustomed to robotic voices=2C simply because text-to-speech systems were=
good enough to produce intelligible speech that was not human-like in its=
sound. Now we can use that same technology to make robotic speech that is=
indistinguishable from human sound robotic again.</p>

<p>A ring modulator has several advantages: It is computationally simple=
=2C can be applied in real-time=2C does not affect the intelligibility of=
the voice=2C and -- most importantly -- is universally =E2=80=9Crobotic s= ounding=E2=80=9D because of its historical usage for depicting robots.</p>

<p>Responsible <a href=3D"https://spectrum.ieee.org/tag/ai-companies">AI c= ompanies</a> that provide <a href=3D"https://spectrum.ieee.org/tag/voice-s= ynthesis">voice synthesis</a> or AI <a href=3D"https://spectrum.ieee.org/t= ag/voice-assistants">voice assistants</a> in any form should add a ring mo= dulator of some standard frequency (say=2C between 30-80 Hz) and of a mini=
mum amplitude (say=2C 20 percent). That=E2=80=99s it. People will catch on=
quickly.</p>

<p>Here are a couple of examples you can listen to for examples of what we= =E2=80=99re suggesting. The first clip is an AI-generated =E2=80=9Cpodcast= =E2=80=9D of this article made by <a href=3D"https://g.co/kgs/FyCQAGX">Goo= gle=E2=80=99s NotebookLM</a> featuring two AI =E2=80=9Chosts.=E2=80=9D Goo= gle=E2=80=99s NotebookLM created the podcast script and audio given only t=
he text of this article. The next two clips feature that same podcast with=
the AIs=E2=80=99 voices modulated more and less subtly by a ring modulato= r:</p>

<h5>Raw audio sample generated by Google=E2=80=99s NotebookLM</h5>

<p><audio style=3D"width: 100%;" controls=3D"controls"> Your browser does=
not support the audio element. </audio></p>

<h5>Audio sample with added ring modulator (30 Hz-25%)</h5>

<p><audio style=3D"width: 100%;" controls=3D"controls"> Your browser does=
not support the audio element. </audio></p>

<h5>Audio sample with added ring modulator (30 Hz-40%)</h5>

<p><audio style=3D"width: 100%;" controls=3D"controls"> Your browser does=
not support the audio element. </audio></p>

<p>We were able to generate the audio effect with a 50-line <a href=3D"htt= ps://spectrum.ieee.org/tag/python">Python</a> script generated by <a href= =3D"https://claude.ai/">Anthropic=E2=80=99s Claude</a>. One of the most we= ll-known robot voices were those of <a href=3D"https://en.wikipedia.org/wi= ki/Dalek">the Daleks from Doctor Who</a> in the 1960s. Back then robot voi=
ces were difficult to synthesize=2C so the audio was actually an actor=E2= =80=99s voice run through a ring modulator. It was set to around 30 Hz=2C=
as we did in our example=2C with different modulation depth (amplitude) d= epending on how strong the robotic effect is meant to be. Our expectation=
is that the AI industry will test and converge on a good balance of such=
parameters and settings=2C and will use better tools than a 50-line Pytho=
n script=2C but this highlights how simple it is to achieve.</p>

<p>Of course there will also be nefarious uses of AI voices. <a href=3D"ht= tps://spectrum.ieee.org/tag/scams">Scams</a> that use <a href=3D"https://s= pectrum.ieee.org/tag/voice-cloning">voice cloning</a> have been getting ea= sier every year=2C but they=E2=80=99ve been possible for many years with t=
he right know-how. Just like we=E2=80=99re learning that we can no longer=
trust images and videos we see because they could easily have been AI-gen= erated=2C we will all soon learn that someone who sounds like a family mem=
ber urgently requesting money may just be a scammer using a voice-cloning=
tool.</p>

<p>We don=E2=80=99t expect scammers to follow our proposal: They=E2=80=99l=
l find a way no matter what. But that=E2=80=99s always true of <a href=3D"= https://spectrum.ieee.org/tag/security">security</a> standards=2C and a ri= sing tide lifts all boats. We think the bulk of the uses will be with popu=
lar voice <a href=3D"https://spectrum.ieee.org/tag/apis">APIs</a> from maj=
or companies -- and everyone should know that they=E2=80=99re talking with=
a robot.</p>

<p><em>This essay was written with Barath Raghavan=2C and originally appea=
red in <a href=3D"https://spectrum.ieee.org/audio-deepfake-fix">IEEE Spect= rum</a>.</em></p>

<p style=3D"font-size:88%">** *** ***** ******* *********** *************<=


<h2 style=3D"font-size:125%;font-weight:bold" id=3D"cg16"><a name=3D"cg16"= >Screenshot-Reading Malware</a></h2>

<p><a href=3D"https://www.schneier.com/blog/archives/2025/02/screenshot-re= ading-malware.html"><strong>[2025.02.07]</strong></a> Kaspersky is <a hre= f=3D"https://www.engadget.com/cybersecurity/kaspersky-researchers-find-scr= eenshot-reading-malware-on-the-app-store-and-google-play-211011103.html">r= eporting</a> on a new type of smartphone malware.</p>

<blockquote><p>The malware in question uses optical character recognition=
(OCR) to review a device=E2=80=99s photo library=2C seeking screenshots o=
f recovery phrases for crypto wallets. Based on their assessment=2C infect=
ed Google Play apps have been downloaded more than 242=2C000 times. Kasper=
sky says: =E2=80=9CThis is the first known case of an app infected with OC=
R spyware being found in Apple=E2=80=99s official app marketplace.=E2=80= =9D</p></blockquote>

<p>That=E2=80=99s a tactic I have not heard of before.</p>

<p style=3D"font-size:88%">** *** ***** ******* *********** *************<=


<h2 style=3D"font-size:125%;font-weight:bold" id=3D"cg17"><a name=3D"cg17"=

UK Is Ordering Apple to Break Its Own Encryption</a></h2>

<p><a href=3D"https://www.schneier.com/blog/archives/2025/02/uk-is-orderin= g-apple-to-break-its-own-encryption.html"><strong>[2025.02.08]</strong></=

The <i>Washington Post</i> is <a href=3D"https://www.washingtonpost.com=

/technology/2025/02/07/apple-encryption-backdoor-uk/">reporting</a> that t=
he UK government has served Apple with a =E2=80=9Ctechnical capability not= ice=E2=80=9D as defined by the 2016 Investigatory Powers Act=2C requiring=
it to break the Advanced Data Protection encryption in iCloud for the ben= efit of law enforcement.</p>

<p>This is a big deal=2C and something we in the security community have w= orried was coming for a while now.</p>

<blockquote><p>The law=2C known by critics as the Snoopers=E2=80=99 Charte= r=2C makes it a criminal offense to reveal that the government has even ma=
de such a demand. An Apple spokesman declined to comment.</p>

<p>Apple can appeal the U.K. capability notice to a secret technical panel=
=2C which would consider arguments about the expense of the requirement=2C=
and to a judge who would weigh whether the request was in proportion to t=
he government=E2=80=99s needs. But the law does not permit Apple to delay=
complying during an appeal.</p>

<p>In March=2C when the company was on notice that such a requirement migh=
t be coming=2C it told Parliament: =E2=80=9CThere is no reason why the U.K=
=2E [government] should have the authority to decide for citizens of the wo= rld whether they can avail themselves of the proven security benefits that=
flow from end-to-end encryption.=E2=80=9D</p></blockquote>

<p>Apple is likely to turn the feature off for UK users rather than break=
it for everyone worldwide. Of course=2C UK users will be able to spoof th=
eir location. But this might not be enough. According to the law=2C Apple=
would not be able to offer the feature to anyone who is in the UK at any=
point: for example=2C a visitor from the US.</p>

<p>And what happens next? Australia has <a href=3D"https://www.homeaffairs= =2Egov.au/about-us/our-portfolios/national-security/lawful-access-telecommun= ications/assistance-and-access-industry-assistance-framework">a law</a> en= abling it to ask for the same thing. Will it? Will even more countries fol= low?</p>

<p>This is madness.</p>

<p style=3D"font-size:88%">** *** ***** ******* *********** *************<=


<h2 style=3D"font-size:125%;font-weight:bold" id=3D"cg18"><a name=3D"cg18"= >Pairwise Authentication of Humans</a></h2>

<p><a href=3D"https://www.schneier.com/blog/archives/2025/02/pairwise-auth= entication-of-humans.html"><strong>[2025.02.10]</strong></a> Here=E2=80=
=99s an <a href=3D"https://ksze.github.io/PeerAuth/">easy</a> system for t=
wo humans to remotely authenticate to each other=2C so they can be sure th=
at neither are digital impersonations.</p>

<blockquote><p>To mitigate that risk=2C I have developed this simple solut=
ion where you can setup a unique time-based one-time passcode (TOTP) betwe=
en any pair of persons.</p>

<p>This is how it works:</p>

<ol><li>Two people=2C Person A and Person B=2C sit in front of the same co= mputer and open this page;

</li><li>They input their respective names (e.g. Alice and Bob) onto the s=
ame page=2C and click =E2=80=9CGenerate=E2=80=9D;

</li><li>The page will generate two TOTP QR codes=2C one for Alice and one=
for Bob;

</li><li>Alice and Bob scan the respective QR code into a TOTP mobile app=
(such as Authy or Google Authenticator) on their respective mobile phones=
;

</li><li>In the future=2C when Alice speaks with Bob over the phone or ove=
r video call=2C and wants to verify the identity of Bob=2C Alice asks Bob=
to provide the 6-digit TOTP code from the mobile app. If the code matches=
what Alice has on her own phone=2C then Alice has more confidence that sh=
e is speaking with the real Bob.</li></ol></blockquote>

<p>Simple=2C and clever.</p>

<p style=3D"font-size:88%">** *** ***** ******* *********** *************<=


<h2 style=3D"font-size:125%;font-weight:bold" id=3D"cg19"><a name=3D"cg19"= >Trusted Execution Environments</a></h2>

<p><a href=3D"https://www.schneier.com/blog/archives/2025/02/trusted-encry= ption-environments.html"><strong>[2025.02.11]</strong></a> Really good --=
and detailed -- <a href=3D"https://dl.acm.org/doi/pdf/10.1145/3634737.364= 4993">survey</a> of Trusted Execution Environments (TEEs.)</p>

<p style=3D"font-size:88%">** *** ***** ******* *********** *************<=


<h2 style=3D"font-size:125%;font-weight:bold" id=3D"cg20"><a name=3D"cg20"= >Delivering Malware Through Abandoned Amazon S3 Buckets</a></h2>

<p><a href=3D"https://www.schneier.com/blog/archives/2025/02/delivering-ma= lware-through-abandoned-amazon-s3-buckets.html"><strong>[2025.02.12]</str= ong></a> Here=E2=80=99s a <a href=3D"https://labs.watchtowr.com/8-million-= requests-later-we-made-the-solarwinds-supply-chain-attack-look-amateur/">s= upply-chain attack</a> just waiting to happen. A group of researchers sear= ched for=2C and then registered=2C abandoned Amazon S3 buckets for about $= 400. These buckets contained software libraries that are still used. Presu= mably the projects don=E2=80=99t realize that they have been abandoned=2C=
and still ping them for patches=2C updates=2C and etc.</p>

<blockquote><p>The TL;DR is that this time=2C we ended up discovering ~150=
Amazon S3 buckets that had previously been used across commercial and ope=
n source software products=2C governments=2C and infrastructure deployment= /update pipelines -- and then abandoned.</p>

<p>Naturally=2C we registered them=2C just to see what would happen -- =E2= =80=9Dhow many people are really trying to request software updates from S=
3 buckets that appear to have been abandoned months or even years ago?=E2= =80=9D=2C we naively thought to ourselves.</p></blockquote>

<p>Turns out they got eight million requests over two months.</p>

<p>Had this been an actual attack=2C they would have modified the code in=
those buckets to contain malware and watch as it was incorporated in diff= erent software builds around the internet. This is basically the SolarWind=
s attack=2C but much more extensive.</p>

<p>But there=E2=80=99s a second dimension to this attack. Because these up= date buckets are abandoned=2C the developers who are using them also no lo= nger have the power to patch them automatically to protect them. The mecha= nism they would use to do so is now in the hands of adversaries. Moreover=
=2C often -- but not always -- losing the bucket that they=E2=80=99d use f=
or it also removes the original vendor=E2=80=99s ability to identify the v= ulnerable software in the first place. That hampers their ability to commu= nicate with vulnerable installations.</p>

<p>Software supply-chain security is an absolute mess. And it=E2=80=99s no=
t going to be easy=2C or cheap=2C to fix. Which means that it won=E2=80=99=
t be. Which is an even worse mess.</p>

<p style=3D"font-size:88%">** *** ***** ******* *********** *************<=


<h2 style=3D"font-size:125%;font-weight:bold" id=3D"cg21"><a name=3D"cg21"= >DOGE as a National Cyberattack</a></h2>

<p><a href=3D"https://www.schneier.com/blog/archives/2025/02/doge-as-a-nat= ional.html"><strong>[2025.02.13]</strong></a> In the span of just weeks=
=2C the US government has experienced what may be the most consequential s= ecurity breach in its history -- not through a sophisticated cyberattack o=
r an act of foreign espionage=2C but through official orders by a billiona=
ire with a poorly defined government role. And the implications for nation=
al security are profound.</p>

<p>First=2C it was reported that people associated with the newly created=
Department of Government Efficiency (DOGE) had <a href=3D"https://bsky.ap= p/profile/wyden.senate.gov/post/3lh5ejpwncc23">accessed</a> <a href=3D"htt= ps://www.nytimes.com/2025/02/01/us/politics/elon-musk-doge-federal-payment= s-system.html">the</a> <a href=3D"https://nymag.com/intelligencer/article/= elon-musk-doge-treasury-access-federal-payments.html">US</a> <a href=3D"ht= tps://therecord.media/union-groups-sue-treasury-over-giving-doge-access-to= -data">Treasury</a> computer system=2C giving them the ability to collect=
data on and potentially control the department=E2=80=99s roughly <a href= =3D"https://fiscal.treasury.gov/fds/">$5.45 trillion</a> in annual federal=
payments.</p>

<p>Then=2C we learned that uncleared DOGE personnel had gained access to <=
a href=3D"https://www.nbcnews.com/politics/national-security/usaid-securit= y-leaders-removed-refusing-elon-musks-doge-employees-acce-rcna190357">clas= sified</a> data from the US Agency for International Development=2C possib=
ly copying it onto their own systems. Next=2C the Office of Personnel Mana= gement -- which holds detailed personal data on millions of federal employ= ees=2C including those with security clearances -- <a href=3D"https://feds= coop.com/opm-email-federal-workforce-lawsuit-server-privacy-security/">was=
</a> <a href=3D"https://www.yahoo.com/tech/elon-musk-seizes-computer-syste=
m-171738117.html">compromised</a>. After that=2C <a href=3D"https://www.re= uters.com/world/us/doge-aides-search-medicare-agency-payment-systems-fraud= -wsj-reports-2025-02-05/">Medicaid and Medicare records</a> were compromis= ed.</p>

<p>Meanwhile=2C only partially redacted names of CIA employees <a href=3D"= https://thehill.com/policy/national-security/5129170-cia-email-employee-id= entities/">were sent</a> over an unclassified email account. DOGE personne=
l are also reported to be <a href=3D"https://www.washingtonpost.com/nation= /2025/02/06/elon-musk-doge-ai-department-education/">feeding</a> Education=
Department data into artificial intelligence software=2C and they have al=
so <a href=3D"https://www.reuters.com/world/us/three-doge-members-raise-ac= cess-concerns-us-energy-department-sources-say-2025-02-07/">started workin= g</a> at the Department of Energy.</p>

<p>This story is moving very fast. On Feb. 8=2C a federal judge <a href=3D= "https://www.reuters.com/legal/us-judge-temporarily-blocks-musks-doge-acce= ssing-payment-systems-2025-02-08/">blocked</a> the DOGE team from accessin=
g the Treasury Department systems any further. But given that DOGE workers=
have already copied data and possibly installed and modified software=2C=
it=E2=80=99s unclear how this fixes anything.</p>

<p>In any case=2C breaches of other critical government systems are likely=
to follow unless federal employees stand firm on the protocols protecting=
national security.</p>

<p>The systems that DOGE is accessing are not esoteric pieces of our natio= n=E2=80=99s infrastructure -- they are the <a href=3D"https://www.lawfarem= edia.org/article/elon-musk-weaponizes-the-government">sinews of government= </a>.</p>

<p>For example=2C the Treasury Department systems contain the technical bl= ueprints for how the federal government moves money=2C while the Office of=
Personnel Management (OPM) network contains information on who and what o= rganizations the government employs and contracts with.</p>

<p>What makes this situation unprecedented isn=E2=80=99t just the scope=2C=
but also the method of attack. Foreign adversaries typically spend years=
attempting to penetrate government systems such as these=2C using stealth=
to avoid being seen and carefully hiding any tells or tracks. The Chinese=
government=E2=80=99s 2015 breach of <a href=3D"https://www.washingtonpost= =2Ecom/world/national-security/chinese-hackers-breach-federal-governments-pe= rsonnel-office/2015/06/04/889c0e52-0af7-11e5-95fd-d580f1c5d44e_story.html"= >OPM</a> was a significant US security failure=2C and it illustrated how p= ersonnel data could be used to identify intelligence officers and compromi=
se national security.</p>

<p>In this case=2C external operators with <a href=3D"https://www.wired.co= m/story/elon-musk-government-young-engineers/">limited experience</a> and=
minimal oversight are doing their work in plain sight and under massive p= ublic scrutiny: gaining the highest levels of <a href=3D"https://talkingpo= intsmemo.com/edblog/musk-cronies-dive-into-treasury-dept-payments-code-bas= e">administrative access</a> and making changes to the United States=E2=80=
=99 most sensitive networks=2C potentially introducing new security vulner= abilities in the process.</p>

<p>But the most alarming aspect isn=E2=80=99t just the access being grante=
d. It=E2=80=99s the systematic dismantling of security measures that would=
detect and prevent misuse -- including standard incident response protoco= ls=2C auditing=2C and change-tracking mechanisms -- <a href=3D"https://www= =2Etheguardian.com/us-news/2025/feb/02/usaid-officials-put-on-leave-musk-dog= e">by</a> removing the career officials in charge of those security measur=
es and replacing them with inexperienced operators.</p>

<p>The Treasury=E2=80=99s computer systems have such an impact on national=
security that they were designed with the same principle that guides nucl=
ear launch protocols: No single person should have unlimited power. Just a=
s launching a nuclear missile requires two separate officers turning their=
keys simultaneously=2C making changes to critical financial systems tradi= tionally requires multiple authorized personnel working in concert.</p>

<p>This approach=2C known as =E2=80=9Cseparation of duties=2C=E2=80=9D isn= =E2=80=99t just bureaucratic red tape; it=E2=80=99s a fundamental security=
principle as old as banking itself. When your local bank processes a larg=
e transfer=2C it requires two different employees to verify the transactio=
n. When a company issues a major financial report=2C separate teams must r= eview and approve it. These aren=E2=80=99t just formalities -- they=E2=80= =99re essential safeguards against corruption and error. These measures ha=
ve been <a href=3D"https://www.wired.com/story/elon-musk-government-young-= engineers/">bypassed or ignored</a>. It=E2=80=99s as if someone found a wa=
y to rob Fort Knox by simply declaring that the new official policy is to=
fire all the guards and allow unescorted visits to the vault.</p>

<p>The implications for national security are <a href=3D"https://techcrunc= h.com/2025/02/05/the-biggest-breach-of-u-s-government-data-is-under-way/">= staggering</a>. Sen. Ron Wyden said his office had learned that the attack=
ers gained <a href=3D"https://securityaffairs.com/173776/security/elon-mus= k-s-doge-granted-full-access-to-sensitive-treasury-systems.html">privilege= s</a> that allow them to modify core programs in Treasury Department compu= ters that verify federal payments=2C access encrypted keys that secure fin= ancial transactions=2C and alter audit logs that record system changes. Ov=
er at OPM=2C reports indicate that individuals associated with DOGE <a hre= f=3D"https://www.rawstory.com/elon-musk-doge-lawsuit/">connected</a> an un= authorized server into the network. They are also reportedly <a href=3D"ht= tps://gizmodo.com/elon-musks-doge-running-highly-sensitive-government-data= -through-ai-report-2000560381">training</a> <a href=3D"https://www.washing= tonpost.com/nation/2025/02/06/elon-musk-doge-ai-department-education/">AI<=

software on all of this sensitive data.</p>

<p>This is much more critical than the initial unauthorized access. These=
new servers have unknown capabilities and configurations=2C and there=E2= =80=99s no evidence that this new code has gone through any rigorous secur=
ity testing protocols. The AIs being trained are certainly not secure enou=
gh for this kind of data. All are ideal targets for any adversary=2C forei=
gn or domestic=2C also seeking access to federal data.</p>

<p>There=E2=80=99s a reason why every modification -- hardware or software=
-- to these systems goes through a complex planning process and includes=
sophisticated access-control mechanisms. The national security crisis is=
that these systems are now much more vulnerable to dangerous attacks at t=
he same time that the legitimate system administrators trained to protect=
them have been <a href=3D"https://www.reuters.com/world/us/musk-aides-loc= k-government-workers-out-computer-systems-us-agency-sources-say-2025-01-31= /">locked out</a>.</p>

<p>By modifying core systems=2C the attackers have not only compromised cu= rrent operations=2C but have also left behind vulnerabilities that could b=
e exploited in future attacks -- giving adversaries such as Russia and Chi=
na an <a href=3D"https://therecord.media/doge-opm-treasury-cybersecurity">= unprecedented</a> <a href=3D"https://cyberscoop.com/musk-doge-opm-treasury= -breach/">opportunity</a>. These countries have long targeted these system=
s. And they don=E2=80=99t just want to gather intelligence -- they also wa=
nt to understand how to disrupt these systems in a crisis.</p>

<p>Now=2C the technical details of how these systems operate=2C their secu= rity protocols=2C and their vulnerabilities are now potentially exposed to=
unknown parties without any of the usual safeguards. Instead of having to=
breach heavily fortified digital walls=2C these parties can simply walk=
through doors that are being propped open -- and then erase evidence of t= heir actions.</p>

<p>The security implications span three critical areas.</p>

<p>First=2C system manipulation: External operators can now modify operati=
ons while also altering audit trails that would track their changes. Secon= d=2C data exposure: Beyond accessing personal information and transaction=
records=2C these operators can copy entire system architectures and secur=
ity configurations -- in one case=2C the technical blueprint of the countr= y=E2=80=99s federal payment infrastructure. Third=2C and most critically=
=2C is the issue of system control: These operators can alter core systems=
and authentication mechanisms while disabling the very tools designed to=
detect such changes. This is more than modifying operations; it is modify=
ing the infrastructure that those operations use.</p>

<p>To address these vulnerabilities=2C three immediate steps are essential=
=2E First=2C unauthorized access must be revoked and proper authentication p= rotocols restored. Next=2C comprehensive system monitoring and change mana= gement must be reinstated -- which=2C given the difficulty of cleaning a c= ompromised system=2C will likely require a complete system reset. Finally=
=2C thorough audits must be conducted of all system changes made during th=
is period.</p>

<p>This is beyond politics -- this is a matter of national security. Forei=
gn national intelligence organizations will be quick to take advantage of=
both the chaos and the new insecurities to steal US data and install back= doors to allow for future access.</p>

<p>Each day of continued unrestricted access makes the eventual recovery m=
ore difficult and increases the risk of irreversible damage to these criti=
cal systems. While the full impact may take time to assess=2C these steps=
represent the minimum necessary actions to begin restoring system integri=
ty and security protocols.</p>

<p>Assuming that anyone in the government still cares.</p>

<p><em>This essay was written with Davi Ottenheimer=2C and originally appe= ared in <a href=3D"https://foreignpolicy.com/2025/02/11/doge-cyberattack-u= nited-states-treasury/">Foreign Policy</a>.</em></p>

<p style=3D"font-size:88%">** *** ***** ******* *********** *************<=


<h2 style=3D"font-size:125%;font-weight:bold" id=3D"cg22"><a name=3D"cg22"=

AI and Civil Service Purges</a></h2>

<p><a href=3D"https://www.schneier.com/blog/archives/2025/02/ai-and-civil-= service-purges.html"><strong>[2025.02.14]</strong></a> Donald Trump and E=
lon Musk=E2=80=99s chaotic approach to reform is upending government opera= tions. Critical functions have been <a href=3D"https://www.theguardian.com= /us-news/2025/feb/05/musk-doge-takeover-usaid">halted</a>=2C tens of thous= ands of federal staffers are being encouraged to <a href=3D"https://www.ws= j.com/lifestyle/careers/federal-workers-accept-buyout-offers-be1c00fb">res= ign</a>=2C and congressional mandates are being <a href=3D"https://thehill= =2Ecom/business/5124133-democrats-bill-treasury-system-musk/">disregarded</a= >. The next phase: The Department of Government Efficiency <a href=3D"http= s://www.nytimes.com/2025/02/03/technology/musk-allies-ai-government.html">= reportedly</a> wants to use AI to cut costs. According to <em>The Washingt=
on Post</em>=2C Musk=E2=80=99s group has started to <a href=3D"https://www= =2Ewashingtonpost.com/nation/2025/02/06/elon-musk-doge-ai-department-educati= on/">run sensitive data</a> from government systems through AI programs to=
analyze spending and determine what could be pruned. This may lead to the=
elimination of human jobs in favor of automation. As one government offic=
ial who has been tracking Musk=E2=80=99s DOGE team told the<em> Post</em>=
=2C the ultimate aim is to use AI to replace =E2=80=9C<a href=3D"https://w= ww.washingtonpost.com/business/2025/02/08/doge-musk-goals/">the human work= force with machines</a>.=E2=80=9D (Spokespeople for the White House and DO=
GE did not respond to requests for comment.)</p>

<p>Using AI to make government more efficient is a worthy pursuit=2C and t=
his is not a new idea. The Biden administration disclosed more than 2=2C00=
0 <a href=3D"https://github.com/ombegov/2024-Federal-AI-Use-Case-Inventory= ">AI applications</a> in development across the federal government. For ex= ample=2C <a href=3D"https://www.dhs.gov/ai/use-case-inventory/fema">FEMA</=

has started using AI to help perform damage assessment in disaster area=

s. The <a href=3D"https://ai.cms.gov/assets/CMS_AI_Playbook.pdf">Centers f=
or Medicare and Medicaid Services</a> has started using AI to look for fra= udulent billing. The idea of replacing dedicated and principled civil serv= ants with AI agents=2C however=2C <em>is </em>new -- and complicated.</p>

<p>The civil service -- the massive cadre of employees who operate governm=
ent agencies -- plays a vital role in translating laws and policy into the=
operation of society. New presidents can issue sweeping executive orders=
=2C but they often have no real effect until they actually change the beha= vior of public servants. Whether you think of these people as essential an=
d <a href=3D"https://www.washingtonpost.com/opinions/interactive/2024/mich= ael-lewis-conclusion-who-is-government/">inspiring</a> do-gooders=2C borin=
g bureaucratic functionaries=2C or as agents of a =E2=80=9C<a href=3D"http= s://www.theatlantic.com/health/archive/2024/11/deep-state-public-health-tr= ump-kennedy/680621/">deep state</a>=2C=E2=80=9D their sheer number and con= tinuity act as ballast that resists institutional change.</p>

<p>This is why Trump and Musk=E2=80=99s actions are so significant. The mo=
re AI decision making is integrated into government=2C the easier change w=
ill be. If human workers are widely replaced with AI=2C executives will ha=
ve unilateral authority to instantaneously alter the behavior of the gover= nment=2C profoundly raising the stakes for transitions of power in democra=
cy. Trump=E2=80=99s unprecedented purge of the civil service might be the=
last time a president needs to replace the human beings in government in=
order to dictate its new functions. Future leaders may do so at the press=
of a button.</p>

<p>To be clear=2C the use of AI by the executive branch doesn=E2=80=99t ha=
ve to be disastrous. In theory=2C it could allow new leadership to swiftly=
implement the wishes of its electorate. But this could go very badly in t=
he hands of an authoritarian leader. AI systems concentrate power at the t= op=2C so they could allow an executive to effectuate change over sprawling=
bureaucracies instantaneously. Firing and replacing tens of thousands of=
human bureaucrats is a huge undertaking. Swapping one AI out for another=
=2C or modifying the rules that those AIs operate by=2C would be much simp= ler.</p>

<p>Social-welfare programs=2C if automated with AI=2C could be redirected=
to systematically benefit one group and disadvantage another with a singl=
e prompt change. Immigration-enforcement agencies could prioritize people=
for investigation and detainment with one instruction. Regulatory-enforce= ment agencies that monitor corporate behavior for malfeasance could turn t= heir attention to=2C or away from=2C any given company on a whim.</p>

<p>Even if Congress were motivated to fight back against Trump and Musk=2C=
or against a future president seeking to bulldoze the will of the legisla= ture=2C the absolute power to command AI agents would make it easier to su= bvert legislative intent. AI <a href=3D"https://www.techpolicy.press/anato= my-of-an-ai-coup/">has the power to diminish</a> representative politics.=
Written law is never fully determinative of the actions of government --=
there is always wiggle room for presidents=2C appointed leaders=2C and ci=
vil servants to exercise their own judgment. Whether intentional or not=2C=
whether charitably or not=2C each of these actors uses discretion. In hum=
an systems=2C that discretion is widely distributed across many individual=
s -- people who=2C in the case of career civil servants=2C usually outlast=
presidencies.</p>

<p>Today=2C the AI ecosystem is dominated by a small number of corporation=
s that decide how the most widely used AI models are designed=2C which dat=
a they are trained on=2C and which instructions they follow. Because their=
work is <a href=3D"https://crfm.stanford.edu/fmti/paper.pdf">largely secr= etive and unaccountable</a> to public interest=2C these tech companies are=
capable of making changes to the bias of AI systems -- either generally o=
r with aim at specific governmental use cases -- that are invisible to the=
rest of us. And these private actors are both vulnerable to coercion by p= olitical leaders and self-interested in appealing to their favor. Musk him= self created and funded xAI=2C now one of the world=E2=80=99s largest AI l= abs=2C with an <a href=3D"https://www.zdnet.com/article/i-tried-xs-anti-wo= ke-grok-ai-chatbot-the-results-were-the-opposite-of-what-i-expected/">expl= icitly ideological</a> mandate to generate anti-=E2=80=9Cwoke=E2=80=9D AI=
and <a href=3D"https://www.wired.com/llm-political-bias/">steer</a> the w= ider AI industry in a similar direction.</p>

<p>But there=E2=80=99s a second way that AI=E2=80=99s transformation of go= vernment could go. AI development could happen inside of transparent and a= ccountable public institutions=2C alongside its continued development by B=
ig Tech. Applications of AI in democratic governments could be focused on=
benefitting public servants and the communities they serve by=2C for exam= ple=2C making it easier for non-English speakers to access government serv= ices=2C making ministerial tasks such as processing routine applications m=
ore efficient and reducing backlogs=2C or helping constituents weigh in on=
the policies deliberated by their representatives. Such AI integrations s= hould be done gradually and carefully=2C with public oversight for their d= esign and implementation and monitoring and guardrails to avoid unacceptab=
le bias and harm.</p>

<p>Governments around the world are demonstrating how this could be done=
=2C though it=E2=80=99s early days. <a href=3D"https://talktothecity.org">= Taiwan</a> has pioneered the use of AI models to facilitate deliberative d= emocracy at an unprecedented scale. Singapore has been a leader in the dev= elopment of <a href=3D"https://www.brookings.edu/articles/how-public-ai-ca= n-strengthen-democracy/">public AI</a> models=2C built <a href=3D"https://= sea-lion.ai">transparently</a> and with <a href=3D"https://www.undp.org/po= licy-centre/singapore/blog/pairing-ai-public-sector-impact-singapore">publ= ic-service use cases</a> in mind. <a href=3D"https://www.canada.ca/en/gove= rnment/system/digital-government/digital-government-innovations/responsibl= e-use-ai/algorithmic-impact-assessment.html">Canada</a> has illustrated th=
e role of disclosure and public input on the consideration of AI use cases=
in government. Even if you do not trust the current White House to follow=
any of these examples=2C U.S. states -- which have much greater contact a=
nd influence over the daily lives of Americans than the federal government=
-- could lead the way on this kind of responsible development and deploym=
ent of AI.</p>

<p>As the political theorist <a href=3D"https://wwnorton.com/books/9781631= 496943">David Runciman</a> has written=2C AI is just another in a long lin=
e of artificial =E2=80=9Cmachines=E2=80=9D used to govern how people live=
and act=2C not unlike corporations and states before it. AI doesn=E2=80=
=99t replace those older institutions=2C but it changes how they function.=
As the Trump administration forges stronger ties to Big Tech and AI devel= opers=2C we need to recognize the potential of that partnership to steer t=
he future of democratic governance -- and act to make sure that it does no=
t enable future authoritarians.</p>

<p><em>This essay was written with Nathan E. Sanders=2C and originally app= eared in <a href=3D"https://www.theatlantic.com/technology/archive/2025/02= /doge-ai-plans/681635/">The Atlantic</a>.</em></p>

<p style=3D"font-size:88%">** *** ***** ******* *********** *************<=


<h2 style=3D"font-size:125%;font-weight:bold" id=3D"cg23"><a name=3D"cg23"= >Upcoming Speaking Engagements</a></h2>

<p><a href=3D"https://www.schneier.com/blog/archives/2025/02/upcoming-spea= king-engagements-43.html"><strong>[2025.02.14]</strong></a> This is a cur=
rent list of where and when I am scheduled to speak:</p>



<li>I=E2=80=99m speaking at <a href=3D"https://boskone.org/">Boskone 6= 2</a> in Boston=2C Massachusetts=2C USA=2C which runs from February 14-16=
=2C 2025. My talk is at 4:00 PM ET on the 15th.</li>

<li>I=E2=80=99m speaking at the <a href=3D"https://www.cl.cam.ac.uk/ev= ents/rossfest/">Rossfest Symposium</a> in Cambridge=2C UK=2C on March 25=
=2C 2025.</li>
</ul>

<p>The list is maintained on <a href=3D"https://www.schneier.com/events/">= this page</a>.</p>


<p style=3D"font-size:88%">** *** ***** ******* *********** *************<=




<p>Since 1998=2C CRYPTO-GRAM has been a free monthly newsletter providing=
summaries=2C analyses=2C insights=2C and commentaries on security technol= ogy. To subscribe=2C or to read back issues=2C see <a href=3D"https://www.= schneier.com/crypto-gram/">Crypto-Gram's web page</a>.</p>

<p>You can also read these articles on my blog=2C <a href=3D"https://www.s= chneier.com">Schneier on Security</a>.</p>

<p>Please feel free to forward CRYPTO-GRAM=2C in whole or in part=2C to co= lleagues and friends who will find it valuable. Permission is also granted=
to reprint CRYPTO-GRAM=2C as long as it is reprinted in its entirety.</p>

<p><span style=3D"font-style: italic">Bruce Schneier is an internationally=
renowned security technologist=2C called a security guru by the <cite sty= le=3D"font-style:normal">Economist</cite>. He is the author of over one do=
zen books -- including his latest=2C <a href=3D"https://www.schneier.com/b= ooks/a-hackers-mind/"><cite style=3D"font-style:normal">A Hacker=E2=80=99s=
Mind</cite></a> -- as well as hundreds of articles=2C essays=2C and acade=
mic papers. His newsletter and blog are read by over 250=2C000 people. Sch= neier is a fellow at the Berkman Klein Center for Internet & Society at Ha= rvard University; a Lecturer in Public Policy at the Harvard Kennedy Schoo=
l; a board member of the Electronic Frontier Foundation=2C AccessNow=2C an=
d the Tor Project; and an Advisory Board Member of the Electronic Privacy=
Information Center and VerifiedVoting.org. He is the Chief of Security Ar= chitecture at Inrupt=2C Inc.</span></p>

<p>Copyright &copy; 2025 by Bruce Schneier.</p>


<p style=3D"font-size:88%">** *** ***** ******* *********** *************<=

<p>Mailing list hosting graciously provided by <a href=3D"https://mailchim= p.com/">MailChimp</a>. Sent without web bugs or link tracking.</p>
<p>This email was sent to: cryptogram@toolazy.synchro.net
<br><em>You are receiving this email because you subscribed to the Crypto-= Gram newsletter.</em></p>

<p><a style=3D"display:inline-block" href=3D"https://schneier.us18.list-ma= nage.com/unsubscribe?u=3Df99e2b5ca82502f48675978be&id=3D22184111ab&t=3Db&e= =3D70f249ec14&c=3D0ce3e98ce0">unsubscribe from this list</a>&nbsp;&nbsp;&nbs= p;&nbsp;<a style=3D"display:inline-block" href=3D"https://schneier.us18.li= st-manage.com/profile?u=3Df99e2b5ca82502f48675978be&id=3D22184111ab&e=3D70f249ec14&c=3D0ce3e98ce0">update subscription preferences</a>
<br>Bruce Schneier &middot; Harvard Kennedy School &middot; 1 Brattle Squa=
re &middot; Cambridge=2C MA 02138 &middot; USA</p>


</body></html>
--_----------=_MCPart_79213603--