Ni,
Now you can compare this here from 2008
with modern AI Laptops for 500-1000 USD:
Google spotlights data center inner workings https://web.archive.org/web/20131019063218/http://news.cnet.com/8301-10784_3-9955184-7.html
There is a striking similarity, only what
once occupied a rack, has now the size
of your plam, all inside one silicon chip:
- Multiple CPU cores on the same chip
- Multiple GPU units on the same chip
- Network on the same chip communication
- Crossbar caches on the same chip
- Disk controllers on the same chip
- Multi channel RAM access on the same chip
Pretty cool!
P.S.: Example such devices with iGPU:
Intel(R) Core(TM) Ultra 7 258V
AMD Ryzen AI 7 350 w/ Radeon 860M
Apple A18 Pro, Darwin Kernel Version 25.5.0
Snapdragon(R) X - X126100 - Qualcomm(R) Oryon(TM) CPU
Mild Shock schrieb:
Hi,
Remember when first all local AI was Python
and PyTorch APIs. And then suddently people strated
using bare metal C/C++ Code. Here is the story:
How it started:
GPT-J or GPT-J-6B is an open-source large
language model (LLM) developed by EleutherAI
in 2021. As the name suggests, it is a
generative pre-trained transformer model
designed to produce human-like text that
continues from a prompt.
https://www.eleuther.ai/
How it was going [Georgi Gerganov]:
So a few days later comes out the LLaMA, I do
some calculations and I figure out rCLOkay, 65
billion parameters. You probably need about
40 gigs of RAM, with 4-bit quantization. So
this can run on a MacBook. Why not do it?rCY
Why I was able to do it so quickly - basically,
for all that I saw itrCOs pretty much GPT-J architecture
with some modifications, like some extra memorization
layers. ItrCOs minor changes. Basically, again, the
existing code for the GPT-J, I just simply
modified it there, it happened pretty quickly.
https://changelog.com/podcast/532
Georgi Gerganov, Bulgarian, now with Hugging
Face, ggml-cann also running on Chinese AI chips.
ggml Manifesto https://github.com/ggml-org/ggml
Bye
Hi,
You might have heard of Rossy Boy the
Fruitcake. But did you ever hear of
the Leiden Snowflakes?
"Consider carefully which tools to use
Some automated tools and their developers
will align with the provisions of this
Declaration, while others will not.
Consider this when deciding which tools
to use, or whether to use them at all.
Also consider whether non-proprietary,
energy-efficient, or small-scale systems
suffice for your task. If not, consider
how preservation of the values articulated
in this Declaration may be worth a
delay in obtaining results." https://leidendeclaration.ai/#choose-tools-carefully
Ok one snowflake weights around 0.01 to
0.05 milligrams. So the weight of this declaration
is like 16 x 0.05 milligrams, ok almost 1 milligram!
BTW: We are currently vibe coding
some AI saftybelts based on the EU KIDS Act,
no AI for KIDS! Its gowing pretty well, especially
because of the new Mandatory OS Age APIs. For
users under 13, AI access is banned without
strict, parental-controlled supervisor accounts.
For ages 13rCo15, AI companion features with persistent
cross-session memory, custom fine-tuning, or
relationship-simulation mechanics are restricted.
Finally the Elite Math Professors are save,
no kid smarter than them!
Bye
P.S.: The EU intiative got a big clap from
America First Policy. App Store Accountability
means Multi-trillion-dollar companies cannot
enter into sophisticated contracts with minors.
They are currently preparing gateways for
"Claude Kids" or "Copilot Education" accounts,
prefering NASDAQ quoted tech over Chinese imports.
Mild Shock schrieb:
Ni,
Now you can compare this here from 2008
with modern AI Laptops for 500-1000 USD:
Google spotlights data center inner workings
https://web.archive.org/web/20131019063218/http://news.cnet.com/8301-10784_3-9955184-7.html
There is a striking similarity, only what
once occupied a rack, has now the size
of your plam, all inside one silicon chip:
- Multiple CPU cores on the same chip
- Multiple GPU units on the same chip
- Network on the same chip communication
- Crossbar caches on the same chip
- Disk controllers on the same chip
- Multi channel RAM access on the same chip
Pretty cool!
P.S.: Example such devices with iGPU:
Intel(R) Core(TM) Ultra 7 258V
AMD Ryzen AI 7 350 w/ Radeon 860M
Apple A18 Pro, Darwin Kernel Version 25.5.0
Snapdragon(R) X - X126100 - Qualcomm(R) Oryon(TM) CPU
Mild Shock schrieb:
Hi,
Remember when first all local AI was Python
and PyTorch APIs. And then suddently people strated
using bare metal C/C++ Code. Here is the story:
How it started:
GPT-J or GPT-J-6B is an open-source large
language model (LLM) developed by EleutherAI
in 2021. As the name suggests, it is a
generative pre-trained transformer model
designed to produce human-like text that
continues from a prompt.
https://www.eleuther.ai/
How it was going [Georgi Gerganov]:
So a few days later comes out the LLaMA, I do
some calculations and I figure out rCLOkay, 65
billion parameters. You probably need about
40 gigs of RAM, with 4-bit quantization. So
this can run on a MacBook. Why not do it?rCY
Why I was able to do it so quickly - basically,
for all that I saw itrCOs pretty much GPT-J architecture
with some modifications, like some extra memorization
layers. ItrCOs minor changes. Basically, again, the
existing code for the GPT-J, I just simply
modified it there, it happened pretty quickly.
https://changelog.com/podcast/532
Georgi Gerganov, Bulgarian, now with Hugging
Face, ggml-cann also running on Chinese AI chips.
ggml Manifesto https://github.com/ggml-org/ggml
Bye
| Sysop: | Amessyroom |
|---|---|
| Location: | Fayetteville, NC |
| Users: | 76 |
| Nodes: | 6 (0 / 6) |
| Uptime: | 55:41:34 |
| Calls: | 1,265 |
| Calls today: | 1 |
| Files: | 1,362 |
| D/L today: |
3 files (2,684K bytes) |
| Messages: | 285,687 |