For most use cases, web search engines are fine. But I am wondering if there are alternative ways to finding information. There is also the enshittification of google and tbh most(free) search engines just give google search result
Obviously, the straight answer is just asking other people, in person or online, in general forums or specialised communities
Libraries are good source too but for those of is that don’t have access to physical libraries, there free online public libraries(I will post the links for those that I found below)
Books in general, a lot of them have reference to outside materials.
So, I been experimenting with an AI chat bot(Le chat), partially as life coach of sorts and partially as a fine tuned web search engine. To cut to the chase, its bad. when its not just listing google top results it list tools that are long gone or just makes shit up. I was hoping it to be a fine tuned search engine, cuz with google, if what you want is not in the top 10 websites, your on your own.
So yeah, that all I can think of. Those are all the routes I can think of for finding information and probably all there is but maybe I missed some other routes.
Complains about the enshittification of search engines, uses an AI chat bot to search things instead. What?
Have you tried using an LLM configured to search the Internet for you? It’s amazing!
Normal search: Loads of useless results, ads, links that are hidden ads, scams, and maybe on like the 3rd page you’ll find what you’re looking for.
AI search: It makes calls out to Google and DDG (or any other search engines you want) simultaneously, checks the content on each page to verify relevancy, then returns a list of URLs that are precisely what you want with summaries of each that it just generated on the fly (meaning: They’re up to date).
You can even do advanced stuff like, “find me ten songs on YouTube related to breakups and use this other site to convert those URLs to .ogg files and put them in my downloads folder.”
Local, FOSS AI running on your own damned PC is fucking awesome. I seriously don’t understand all the hate. It’s the technology everyone’s always wanted and it gets better every day.
LLMs still hallucinate what content is on the pages they search. It’s not great.
It depends on the size of the content on the page. As long as it’s small enough to be contained within the context window, it should do a good job.
But that’s all irrelevant since the point of the summary is just to give you a general idea of what’s on the page. You’ll still get the actual title and whatnot.
Using an LLM to search on your behalf is like using
grepto filter out unwanted nonsense. You don’t use it like, “I’m feeling lucky” and pray for answers. You still need to go and open the pages in the results to get at what you want.
checks the content on each page to verify relevancy,
No it can’t do that. It’s an LLM, it can only generate the next word in a sequence.
Also this doesn’t solve OPs problem at all. If it’s in the top 10 results on a major search engine then anyone can find it in minimal time.
Fucking AI bros being like I remade looking at 10 google links but this time it burns down a forest and tells me what a genius I am for asking.
Explain to me which forest burns down when I run an AI on my local computer that uses the same power (or less) as running a video game?
AI / LLMs aren’t evil or unethical or immoral - commercializing them into enormous behemoths that eat resources 24x7 is.
Even local models are trained on stolen art and content. That’s the immoral part.
No one seems to get this part.
There’s many models that use open source training sets and weights. You can choose them.
AI models aren’t trained on anything “stolen”. When you steal something, the original owner doesn’t have it anymore. That’s not being pedantic, it’s the truth.
Also, if you actually understand how AI training works, you wouldn’t even use this sort of analogy in the first place. It’s so wrong it’s like describing a Flintstones car and saying that’s how automobiles work.
Let’s say you wrote a book and I used it as part of my AI model (LLM) training set. As my code processes your novel, token-by-token (not word-by-word!), it’ll increase or decrease a floating point value by something like 0.001. That’s it. That’s all that’s happening.
To a layman, that makes no sense whatever but it’s the truth. How can a huge list of floating point values be used to generate semi-intelligent text? That’s the actually really fucking complicated part.
Before you can even use a model you need to tokenize the prompt and then perform an inference step which then gets processed a zillion ways before that .safetensors file (which is the AI model) gets used at all.
When an AI model is outputting text, it’s using a random number generator in conjunction with a word prediction algorithm that’s based on the floating point values inside the model. It doesn’t even “copy” anything. It’s literally built upon the back of an RNG!
If an LLM successfully copies something via it’s model that is just random chance. The more copies of something that went into its training, the higher the chance of it happening (and that’s considered a bug, not a feature).
There’s also a problem that can occur on the opposite end: When a single set of tokens gets associated with just one tiny bit of the training set. That’s how you can get it to output the same thing relatively consistently when given the same prompt (associated with that set of tokens). This is also considered a bug and AI researchers are always trying to find ways to prevent this sort of thing from happening.
As much as I understand your hate for LLMs, this is wrong.
No it can’t do that. It’s an LLM, it can only generate the next word in a sequence.
Your knowledge is out of date, friend. These days you can configure an LLM to run tools like
curl,nmap,ping, or even write then execute shell scripts and Python (though, in a sandbox for security).Some tools that help you manage the models are preconfigured to make it easy for them to search the web on your behalf. I wouldn’t be surprised if there’s a whole ecosystem of AI tools just for searching the web that will emerge soon.
What Mozilla is implementing in Firefox will likely start with cloud-based services but eventually it’ll just be using local models, running on your PC. Then all those specialized AI search tools will become less popular as Firefox’s built-in features end up being “good enough”.
I understand how agents work. I’m just saying it cannot “verify relevancy”. That’s a qualitative assessment that an LLM is incapable of doing. The scripts and regex that form the backbone of the “agent” can absolutely download a webpage and add the text contents to the context. But after that it’s just random bullshit.
It does way more than that.
I have it write scripts in 30 seconds that would take me 2 days to write and verify.
I can quickly parse through what it writes (takes me about a minute) to verify it hasn’t done anything wonky, then test it in a VM I use for testing my own scripts.
It does this because the question I ask is very clear and explicit: exact script language/version, exact input, exact output, how the script should flow, what commenting should look like. It takes me about 1 minute to write a good question like this.
I’ve setup Projects in it with specific rules so I don’t have to state those rules every time - I have one for each scripting language. I’ve saved that definition so when I update the rules I have a local definition for reuse or share with my peers.
For informational searches I have it provide source links automatically - I have a lot of general knowledge so whenever it produces something that doesn’t sit right I will steelman the information. It’s surprising what it can come up with this way.
My friends say “you like to argue with it” - well sometimes arguing is necessary.
Haha, downvoters. What a joke.
Agreed, a locally running LLM is an amazing choice.
is there a non- to middling-tech savvy option for a local ai?
Not at this point, no. Not unless you know how to setup/manage docker images and have a GPU with at least 16GB of VRAM.
Also, if you’re not using Linux forget it. All the AI stuff anyone would want to run is a HUGE pain in the ass to run on Windows. The folks developing these models and the tools to use them are all running Linux. Both on their servers and on their desktops and it’s obvious once you start reading the
README.mdfor most of these projects.Some will have instructions for Windows but they’ll either be absolutely enormous or they’ll hand wave away the actual complexity, “These instructions assume you know the basics of advanced rocket science and quantum mechanics.”
i aupreciate this, thank you
I have been really enjoying Kagi, unfortunately it’s a paid service. Kagi allows you to specify the source of your search results. For example, I can limit my search to the Fediverse or Usenet, etc.
That’s not unfortunate to be honest. It’s how they can make a search engine that is both good and doesn’t harvest user data or rely on ads.
I’m old enough that the first kind of “web search” that I used were manually compiled website directories. Some are still around, maybe try a few and see if you like them. Come to think of it, it’s probably still a fun way of exploring websites around a subject. Here’s the current incarnation of one of the most prominent ones:
Thank you for the link, had a conversation the other day talking about what our game plan is when search engines become even more useless. I feel we really go full circle with this one.
Here is an active codeberg repo curating german Amazon alternatives: https://codeberg.org/phranck/Amazon-Alternativen
Man, physical web directories! I wasn’t “around” for them but it’s wild to think the web was that small once. I was born in the mid 80s but didn’t even hear the word internet until around 1996.
I do remember webrings though. That was one way to discover new websites before algorithms took over. Not great for research probably but a fun way to explore.
Webrings.
Kagi.
Ask tech literate humans.
Chatbots can’t think or tell if anything is correct. They can generate fake data and there is literally no amount of “tuning” to fix that. For the love of the planet and your sanity, please never touch one again.
I find duck duck go works better than Google now. I also find perusing Wikipedia will often give me the information I need or point me in a better direction.
The chatbot isn’t the issue here. It’s the user treating it like a reliable source of information.
It’s a large language model - not a large knowledge model. It gets plenty of stuff right, but that’s not because it actually “knows” anything - it’s just trained on a massive pile of correct information.
People trash it for the times it gets things wrong, but it should be the other way around. It’s honestly amazing how much it gets right when you consider that’s not even what it’s built to do.
It’s like cruise control that turns out to be a surprisingly decent driver too.
I’m fully aware of the technology behind it. I trash it because it’s resource sucking, planet burning trash that serves no real purpose.
The technology is inherently flawed and fills no niche because you can never trust anything from it. Even if it’s right 9/10 times that tenth time can, and has, killed people.
It’s a chatbot. You talk to it, and it responds in natural language. That’s exactly what it’s designed to do - and it does it exceptionally well, far better than any system we’ve had before.
Faulting it for being untrustworthy just shows most people don’t actually understand this tech, even though they claim they do. Like I said before: it’s a large language model - not a large knowledge model.
Expecting factual answers from it is like expecting cruise control to steer your car. When you end up in the ditch, it’s not because cruise control is some inherently flawed technology with no purpose. It’s because you misused the system for something it was never designed to do.
If you just Google something like “health effects of hibiscus,” you’ll find a mixture of information too. Most people probably can’t tell which claims are well researched and which ones aren’t.
You’ll be left with a mixed bag, but reading all that takes more time than it takes to read an equally flawed summary you get from a gas powered AI. From a convenience perspective, I can understand why some people might prefer an LLM. From a reliability perspective, I can’t favour either option. Regardless, the difference in environmental impact should be clear to everyone.
We should exact mandatory media literacy classes alongside killing LLMs. I know Finland has them and from what I can tell it’s been a resounding success.
Careful with the whole chatbot-lifecoach thing. People are developing psychosis from being that personal with those things. Don’t let it give you affirmations
LLMs are designed to agree to whatever to say (for the most part) so you could be dead ass wrong and it’ll be like, “you’re absolutely right!”
Which is bad. But you should see some of the shit chatgpt said to Stein-Erik Soelberg. That shit went way beyond just yes-anding misinformation. Dude got straight up groomed
Which is why (for certain things) after it’s provided an answer, I push back and argue a counterpoint, to see what it comes up with.
You can try to not use a search engnie at all, but the Web relies a lot on them. If that’s really what you want, you’re limited to surfing. Go to a site, follow links, and hope you get where you need to go. Surfing is great for finding things you didn’t know you were looking for, and not so great for finding particular things you need information on. For that, either try whatever libraries you have access to or cave and use a search engine.
There are plenty of search engines that aren’t Google. Some don’t use the same search index (big list of web pages that it shows you when you search stuff). I currently use Qwant as my primary engine, and before that i used Startpage. I’m also fond of Marginalia when a big, normal engine isn’t giving me helpful results.
Or you can ask people, like you said. Forums, chatrooms, whatever Lemmy is, maybe email somebody if you think they know stuff about whatever field you’re interested in. Or you could sit down in a coffee shop with a sign that says “Tell me about electrical engineering” or whatever it is you want to know, and see how that goes.
Wikipedia is good, if you don’t mind that it has a search bar with an index consisting of its own pages. You shouldn’t trust everything you read there, but every good article (and most bad ones) cites sources, and you can follow those citations.
I don’t know if DMoz is still out there, but there are web directories in the world. Also, hopping into IRC and asking channel relevant questions sometimes helps.
My first port of call when wanting to find true facts about something is Wikipedia as there’s most always a source attached to any claims. I realised that often I’d search something just to look at the quick facts on the side taken from wikipedia or just to open the page, so I’ve cut out the middleman
There’s a paid search engine that some techies use, can’t remember what it’s called.
Probably Kagi. Everyone that uses it swears by it, in my experience.
I tried it for a bit (until the free searches ran out), and it’s genuinely good. But I don’t use search engines enough to justify paying subscription.
I’m paying for it and can confirm that I swear by it.
Same here. Absolutely worth it. That and I migrated over to Proton for email too
It’s called Kagi, I am using the 3 month free trial and it’s genuinely amazing, sadly they use Google Cloud and sometimes I can’t use it with a VPN and without a VPN it’s sanctioned for me because I am in Iran
Off topic but what’s it like in Iran right now? If what I’m seeing in the news is correct then things are absolutely going off the rails over there atm
For the lower class like me the economy is pretty bad, you have to save up like years for expensive stuff, the protests have kinda died down and it’s back to “normal” which is still pretty bad
Damn. What were the protests like? I heard people were getting killed?
Very bad, they shoot protesters mercilessly in Tehran and they aim for the eye, they put everyone in bodybags and lay them on the streets, in Ardabil they run over protesters with armored cars and now everyone is afraid to go out and reasonably so, they also take money for the bullets from the family then deliver the bodies.
Jesus Christ, I don’t even know what to say. That’s absolutely horrific. I hope you and your family are safe.
Use ecosia.org They are testing their new search index! If you use it you’ll be helping nature, helping fine tune new search engine, etc
Kagi if you’re OK with US based tech, Good if you want EU based tech
That’s pretty much it. I don’t think there are other ways.
Sorry for the tangent, but your post reminded me of Herodotus and his book “Histories”. If anyone reading this don’t know who that is, he’s called “The Father of History” for being the first (known) historian writing down events and history.
If you read his book, it’s full of “he said that, she mentioned this, I heard about, etc”. It’s an interesting experience compared to reading modern books, because modern books reference each other and won’t bother you with where they got that info in the text itself, they’ll just give you the sources at the bottom of the page or at the end of the book. Herodotus didn’t have that, he had to rely on what people said.
This resulted in some interesting accounts. For example, he talks about enormous “ants” that were about the size of foxes, lived in the hills, and carried away piles of sand that contained gold dust, which the locals collected and turned into wealth.
There’s some theories that he was likely talking about marmots, but we’ll never know for certainty. It may have been him just misinterpreting accounts, or maybe it was just someone who pulled his leg and he believed them.
Where I’m getting at, every book/article/etc we have is actually just writing down what someone else said/wrote with new insights. It’s easy to forget that nowadays with modern books and articles, “Histories” is a reminder of that fact.
It depends on the subject. For some things I use search engines, for some things I’ll just google for books or research on the subject matter and obtain pdf’s for those, some stuff on relevant internet fora, and it used to be that youtube had some good info, though lately it’s overrun by AI slop.
