Artificial Intelligence confessions
I sometimes use external AI services that do not require registration. I do it through the Tor Browser but they require JS enabled and, in any case, they are SaSS and consume enormous amounts of resources.
So these services have many bad things on freedom and environment but I sometimes use them (I only know two services I can use through Tor Browser without registering). In my case, despite I often have to go through lots of captchas, these services sometimes help me to save lots of time (and resources too) on some specific tasks.
I would like to know about your experiences with these services, if any, and share links, ways to connect, opinions, etc. But not sure if, despite being in the Troll Lounge were offtopic conversations are allowed, discussing about SaSS services could be going too far. If you think that is the case, please let me know, I will happily understand.
I am also considering to locally run some AI software on my own server, but I do not know about the resources it may require to obtain results comparable to those I get from the SaSS services I know. I quickly tried once on my computer, but the computer could not cope and crashed.
You need enough RAM to store the billions of parameters of an LLM. Some models are "distilled" and require less RAM. I installed Alpaca from Flathub:
$ sudo apt install flatpak
$ flatpak remote-add --subset=floss flathub https://flathub.org/repo/flathub.flatpakrepo
$ flatpak install flathub com.jeffser.Alpaca
In Alpaca, I then added DeepseekR1 with 14B parameters and it takes less than 13 GB of RAM.
I asked the LLM something to measure: "What types of recurrence relation can be solved using generating functions?"... and it literally "thought" for tens of minute (I do not have a discrete GPU that would accelerate the computation, because those rely on non-free software)! I had never experienced that before, but the question is truly hard and the answer was OK, despite the use of a distilled model.
13GB of ram huh... that is a hell of a lot.
I am glad I don't plan to use AI... that is an insane waste of resources. Although I suppose it might have some purposes. But I would say most of the uses that are ethical are just for fun.
13GB of ram huh... that is a hell of a lot.
Well, that is relative. The non distilled Deepseek R1 model has 671 billions parameters and you probably need one of the servers with at least 512 GB of RAM behind the link I gave to amuza. On the other extreme, the smallest distilled version of Deepseek R1 has only 1.5 billion parameters. Looking at the list in Alpaca, I discover a related fined-tuned version, called DeepScaleR, with 1.5 billion parameters too: https://pretty-radio-b75.notion.site/DeepScaleR-Surpassing-O1-Preview-with-a-1-5B-Model-by-Scaling-RL-19681902c1468005bed8ca303013a4e2
I am installing it now, to test how faster but worse the answer to the hard question I made yesterday (I did not realize at the time, it would take that much time and I did not clock).
Still, I bet this AI functionality requires a fairly powerful computer with a lot of ram.
Processor probably has to be either A:
Something like the TALOS 2 and/or a desktop computer in general.
or B:
It has to be like beyond like an eighth gen processor. I guess my point is, neither gnuboot or canoeboot can use such a functionality.
Although there are libreboot and coreboot devices new enough to do this I suppose.
Even older laptops such as the T400, the T500 or the X200 can have 8GB of RAM, right? That should be enough to use DeepScaleR since, as I wrote, even using it through Alpaca, less than 5GB of RAM is required. The answer time will be longer though, because of the older CPU and the DDR3 RAM. And there are even more distilled models. Qwen3 has a 0.6-billion-parameter version and another one with 1.7 billions. And, yes, their weights are licensed, like Deepseek R1's, licensed under a free software license (the Apache 2 license).
Oh, you said 13GB of ram that's why I said that and it sounded like the AI would use a lot of cpu power I thought too.
Nevermind then.
Although I wonder how long it would take on a sandy bridge vs newer stuff though.
Also wonder how much heat it would create too.
I mean it sounds like it uses a lot of resources.
Although I wonder how long it would take on a sandy bridge vs newer stuff though.
Well, you can try if you have such a machine. I don't.
Also wonder how much heat it would create too.
Quite a lot: all cores are used. During winter in a cold country (unlike mine), it is not a waste though.
> I asked the LLM something to measure: "What types of recurrence relation can be solved using generating functions?"
Have you ever asked that to a SaSS AI?
If so, how long did it take to give you a similar answer?
I do not do SaSS. The answer certainly comes much faster because the server is probably one of those: https://en.wikipedia.org/wiki/Nvidia_DGX
With the DeepSclaleR model, running in Alpaca, less than 5 GB of RAM are required (including the RAM to run Alpaca itself) and the same question ("What types of recurrence relation can be solved using generating functions?") is answered in ~6 minutes. The answer is not as good as that of Deepseek R1 with 14 billions parameters. It says that the solvable types "include both linear homogeneous and nonhomogeneous" recurrence relations, which is true because of the choice of verb, "include", but kind of deceptive: it does not mention that some non-linear recurrence relations are solvable in this way, for instance to find a closed-form expression for Catalan numbers, as DeepSeek R1 answers.
I asked DeepSeek R1 the same question again, in a new conversation, to clock the time it takes: ~40 minutes. I repeat: the question is particularly hard (I had never seen such times) and everything runs on a six-year-old CPU (an i7-10510U though).
If you are interested in the answers (and the preceding thoughts, actually written after in the file) of the two models regarding the question, they are attached.
| Attachment | Size |
|---|---|
| DeepScaler-answer.txt | 7.84 KB |
| Deepseek R1-14B-answer.txt | 14.19 KB |
your i7-10510U took 40 minutes?
I wonder how long sandy bridge would take then.
I posited your question to Dickduckgo's ai - Mistral version. Uploaded the answer which came rapidly, maybe 2 seconds. How does it compare?
| Attachment | Size |
|---|---|
| FirstAIQuestionFromMagicBanana.txt | 2.75 KB |
I have tried running ollama with my T400 (4GBs of RAM). The models I can run, are either too slow (CPU only) or... dumb.
I suppose with a higher end machine the experience would be more in line with what Magic Banana mentioned, better replies but still running slow (due to CPU only).
I don't use AI often anyways, more for fun experimenting really. IN my workplace it is being heavily implemented though.
I do bring a share of a website though that people might want to use, since it works well with Tor Browser (no captchas!), no registration required, has many free models available, and only minimal JS is required (meaning you only need to allow the main website JS in NoScript, other than that you can use it with the security slider up to Safest, unlike many other websites I checked that give a lot of errors and require third-party JS). It's not perfect but so far the best solution I have found: www.hotbot.com
As for www.duck.ai I don't understand, sometimes it works with the Tor Browser some others it doesn't! Maybe I should ask the AI about it :-P
If anyone has similarly clean websites that would work in these conditions, I would be happy to know about :)
Apologies, I correct myself. Just checked the website and you also need to allow "jsdelivr.net" for it to work.
I haven't used it in a while, maybe I misremembered or they just changed it lately.
Still, it's the cleanest experience I have had in AI websites.
Open for suggestions from others :)
I never use any AI of SaaSS.