From Newsgroup: comp.ai

I have a system with a very high-end CPU (for a workstation), a 12th
generation Intel i9, but a low-end GPU (Radeon RX470 with 6 gb of RAM).

Is it possible to do any sort of AI training on that setup, say if you're willing to process at 5% of an nVidia GPU's speed? I just want to play with
it, see if it's worth investing more money, at this point. I don't mind, if, say, training a LORA takes an hour per photo, or even 6 hours per photo.

Sorry if I mangled any terminology, this is not my field and I'm (as you guessed by now) just starting to learn it.

Thanks.
--
Carl Fink carl@finknetwork.com
https://reasonablyliterate.com https://nitpicking.com
If you want to make a point, somebody will take the point and stab you with it.
-Kenne Estes

--- Synchronet 3.21a-Linux NewsLink 1.2

From Newsgroup: comp.ai

Greetings.

On 2024-11-28 13:05, Carl Fink wrote:

I have a system with a very high-end CPU (for a workstation), a 12th generation Intel i9, but a low-end GPU (Radeon RX470 with 6 gb of RAM).

Is it possible to do any sort of AI training on that setup, say if you're willing to process at 5% of an nVidia GPU's speed? I just want to play with it, see if it's worth investing more money, at this point. I don't mind, if, say, training a LORA takes an hour per photo, or even 6 hours per photo.

Yes, it's absolutely possible to train generative AI on a CPU rather
than a GPU, but whether this is feasible depends entirely on what sort
of model you are expecting to end up with, how much training time you're willing to tolerate, and how much RAM you have available. Without
further details on your requirements it's difficult to make any specific recommendations. I've seen a few reports comparing CPU and GPU training
of LoRA models for Stable Diffusion, for example, that indicate that CPU training can require four times more memory and/or forty times more
time. If you don't already have the requisite GPU hardware and don't
want to buy it yourself, it might be economical to rent the processing
power from a cloud service.

Regards,
Tristan
--
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Tristan Miller
Free Software developer, ferret herder, logologist
https://logological.org/ =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

--- Synchronet 3.21a-Linux NewsLink 1.2

From Newsgroup: comp.ai

On 2024-12-01, Tristan Miller <psychonaut@nothingisreal.com> wrote:

Yes, it's absolutely possible to train generative AI on a CPU rather
than a GPU, but whether this is feasible depends entirely on what sort
of model you are expecting to end up with, how much training time you're willing to tolerate, and how much RAM you have available. Without
further details on your requirements it's difficult to make any specific recommendations. I've seen a few reports comparing CPU and GPU training
of LoRA models for Stable Diffusion, for example, that indicate that CPU training can require four times more memory and/or forty times more
time. If you don't already have the requisite GPU hardware and don't
want to buy it yourself, it might be economical to rent the processing
power from a cloud service.

Can you link to those reports, or tell me where to search (e. g. a journal name)?

This box has 48 GB of RAM. As I wrote originally, I'm fine with long
processing times, although 40x seems very long.

Thank you.
--
Carl Fink carl@finknetwork.com
https://reasonablyliterate.com https://nitpicking.com
If you want to make a point, somebody will take the point and stab you with it.
-Kenne Estes

--- Synchronet 3.21a-Linux NewsLink 1.2

From Newsgroup: comp.ai

Dear Carl,

On 2024-12-01 19:23, Carl Fink wrote:

Can you link to those reports, or tell me where to search (e. g. a journal name)?

This box has 48 GB of RAM. As I wrote originally, I'm fine with long processing times, although 40x seems very long.

I'm afraid I was reporting only half-remembered results from the last
time I looked into the question, which would have been months ago. I
did a quick web search just now and came up with a couple queries from
the Kohya's GUI GitHub project that roughly accord with my recollection:

https://github.com/bmaltais/kohya_ss/discussions/679 https://github.com/bmaltais/kohya_ss/issues/2632

The first of these claims CPU training time of about 40|u, and the second claims 4|u for both time and memory. They both refer to Stable Diffusion.

There's also this LoRA finetuning guide for LLaMA that provides detailed
CPU time and memory metrics for various models: https://rentry.org/cpu-lora

Regards,
Tristan
--
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Tristan Miller
Free Software developer, ferret herder, logologist
https://logological.org/ =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

--- Synchronet 3.21a-Linux NewsLink 1.2

From Newsgroup: comp.ai

On 2024-12-02, Tristan Miller <psychonaut@nothingisreal.com> wrote:

I'm afraid I was reporting only half-remembered results from the last
time I looked into the question, which would have been months ago. I
did a quick web search just now and came up with a couple queries from
the Kohya's GUI GitHub project that roughly accord with my recollection:

https://github.com/bmaltais/kohya_ss/discussions/679 https://github.com/bmaltais/kohya_ss/issues/2632

The first of these claims CPU training time of about 40|u, and the second claims 4|u for both time and memory. They both refer to Stable Diffusion.

There's also this LoRA finetuning guide for LLaMA that provides detailed
CPU time and memory metrics for various models: https://rentry.org/cpu-lora

Thank you!
--
Carl Fink carl@finknetwork.com
https://reasonablyliterate.com https://nitpicking.com
If you want to make a point, somebody will take the point and stab you with it.
-Kenne Estes

--- Synchronet 3.21a-Linux NewsLink 1.2