• I added an agora current events board to contain discussions of political and current events to that category. This was due to a increase support for a separate board for political talk.

Search Engines Suck Now - What do we do about it?

Regal

Well-Known Traveler
Joined
Nov 20, 2022
Messages
399
Reaction score
1,575
Awards
132
Been using Kagi for about 6 months now. Better search results, better privacy, and the team behind it seem to give a shit about the product.

Kagi is the last good search engine before the entire idea of a search engine is scrapped for AI. Normies don't care about websites. They only care about the information on websites so if they can skip the "web surfing" AI will become the dominant method for using the internet. For better or worse.
 

K0WLOON

Well-Known Traveler
Joined
Feb 3, 2020
Messages
381
Reaction score
876
Awards
95
Website
kowl00n.neocities.org
Been using Kagi for about 6 months now. Better search results, better privacy, and the team behind it seem to give a shit about the product.

Kagi is the last good search engine before the entire idea of a search engine is scrapped for AI. Normies don't care about websites. They only care about the information on websites so if they can skip the "web surfing" AI will become the dominant method for using the internet. For better or worse.
The paying for search is depressing. It is, in effect, a sign of the ongoing death of the utopian ideals of the early web. This death is in no way a good thing and is likely to be a blow to the foundations of any actual "free" aspects of the modern world that the ones have managed to hold onto or create and maintain in the face of the increasing centralization of power and control in both the public and private speheres led by the state and industry corpos respectively.

Ones are, of course, only interested in the instrumental use-value of the thing tho so it will continue to thrive as an ersatz replacement for the old web as the other ones continue the march towards being slop troughs and all-you-can eateries of nonsense.


So it goes, that grass hut in the woods begins to look more and more appealing with every passing day
 
Virtual Cafe Awards

cloudfair

Guy
Joined
Jan 6, 2025
Messages
71
Reaction score
281
Awards
41
What can I say, you talk very confidently about things that you don't really seem to have a clue about.
Brother, take a chill pill. its also very ironic you say this
It is, because it's spam. Search engine optimisation can be done by improving site and content or it can be done in other ways. Spam is "in other ways".
"SEO spam is the problem because its spam". This does not make any sense. SEO spam is irrelevant when it comes to googles shitty search results.
Google doesn't define what search engine optimisation is. It also doesn't define what the methods for search engine optimisation there are. But it's true that SEO mainly targets Google.
Yes they do. The "search engine" part of SEO might as well refer to google, because if google changes something with regards to ranking pages then that is the new algorithm to be optimized. I am not sure if you realize how insignificant other engines are in comparison
Incorrect. If it was easy, LLM spam wouldn't have good position on Google. Google doesn't want LLM spam as results because that makes people do shit like "[search term] >redditcostanzayeahrightsmirk" which means that Google's search engine is not good.
⠀ ⠀I won't go on to guess why it's difficult because I don't have any better information on the matter than you, but consider that LLM spam is generated with enormous amount of computational power to appear as natural language.
Have you considered that maybe the reason LLM spam is prominent is because google wants it to be? Googles goal above all else is to make money not to have the most quality search results. The vast majority of their revenue comes from sponsored links, google ads shown with results, and the google ads on those LLM spam sites. A quality or niche website that many may consider a great search result will likely not get returned if it doesnt have any ads, its not profitable for google to serve a site with no ads. In fact, I would bet that the type of people that post quality information on the web would be more likely to be against running ads, thus they dont get traffic.
Google doesn't have a copy of every website they have scraped. Why would they? For that purpose go see archive.org.
Well they absolutely have a copy of ever page in the search index, or else they wouldnt be able to send it to your computer lol. But as for storing older versions of pages and what not, today data is power so they absolutely would. Google was training deep learning models all the way back in 2010 and part of the reason they could do this was because of their massive collection of user generated content. havent you noticed all of the hype around scraping websites or >redditcostanzayeahrightsmirk paywalling their API when they realized how much free value people were getting for LLMs? Or perhaps you notice there are 40+ robots scraping this website at all times? Google probably has 10 pages for every 1 on archive.org
Which could literally be anything that searches the web better. Sure, if some LLM-kind of solution gives responses to random questions better, it can kill off some of the traffic Google gets. But LLMs don't search the web, which is what Google the search engine exists for.
if you think some random startup could rival google search, you probably do not understand what all goes into a search engine. Just the storage devices for the index will run you a billion dollars
 

tuit

A nuisance by the grace of God
Joined
Jun 3, 2025
Messages
46
Reaction score
142
Awards
34
its also very ironic you say this
I'm just going to come out with the fact that I do this for a living, admittedly among other things.
"SEO spam is the problem because its spam". This does not make any sense. SEO spam is irrelevant when it comes to googles shitty search results.
At this point I may as well stop reading because you don't seem to understand what "optimisation" means.
⠀⠀From Merriam-Webster:
an act, process, or methodology of making something (such as a design, system, or decision) as fully perfect, functional, or effective as possible
Or perhaps what this means in the context of a search engine is unclear to you. In the context of search engines, optimisation means achieving the highest possible position in the search results.
Yes they do. The "search engine" part of SEO might as well refer to google, because if google changes something with regards to ranking pages then that is the new algorithm to be optimized. I am not sure if you realize how insignificant other engines are in comparison
In the context of this discussion yes. Google does not define the act nor the methods of search engine optimisation. Google kindly asks you to optimise in certain ways. This is because again and again Google finds itself fighting with people abusing their search engine to optimise the search engine result position in ways that make Google worse as a product.
⠀⠀Here's what they have to say about LLMs in the context of SEO by the way.
Have you considered that maybe the reason LLM spam is prominent is because google wants it to be?
Yes, and found it an utterly pseud take.
Googles goal above all else is to make money not to have the most quality search results. The vast majority of their revenue comes from sponsored links, google ads shown with results, and the google ads on those LLM spam sites.
True, but they make the most money by having the most quality search results. If you're interested in the money, I suggest you take a look on Alphabet Q1 2025 financials. Maybe you're interested in Alphabet's yearly report 2024.
⠀ ⠀ Google Ads on LLM spam sites are not so large of a source for income that it justifies butchering their ~equal (to their whole other ads network) marketing platform that is the search engine.
A quality or niche website that many may consider a great search result will likely not get returned if it doesnt have any ads, its not profitable for google to serve a site with no ads.
False. Spending on Google Search ads improves performance of search engine
...you get more traffic to your website.
⠀⠀Google ads on your page worsens search engine position in a similar way as having Google's Tag Manager running on your website: more JavaScript to run.
In fact, I would bet that the type of people that post quality information on the web would be more likely to be against running ads, thus they dont get traffic.
Showing ads is not the only way to make money on the web and showing Google ads is in fact a comparatively bad way to make money on the web. I don't show any Google ads and none of my clients that are involved with me on the matter show any Google ads.
⠀⠀I do advertise on Google ads and much more than that I advertise on the search.
Well they absolutely have a copy of ever page in the search index, or else they wouldnt be able to send it to your computer lol.
lol, they don't send it to your computer any more than I am sending a video of a man spinning his penis while getting fucked by another man to the sound of "You Spin Me Round (Like a Record)". I'm simply linking you to it. This video will be saved on my computer until the next time my browser seems it fit to clear the cache.
But as for storing older versions of pages and what not, today data is power so they absolutely would.
They have no use in storing the full page data. They have no use in storing the full page content either.
Google was training deep learning models all the way back in 2010 and part of the reason they could do this was because of their massive collection of user generated content.
True, but they don't need the full page data or content to do this.
Or perhaps you notice there are 40+ robots scraping this website at all times? Google probably has 10 pages for every 1 on archive.org
I can't know of the bots scraping Agora because I'm not the webmaster. Neither can you.
if you think some random startup could rival google search, you probably do not understand what all goes into a search engine. Just the storage devices for the index will run you a billion dollars
Every company used to—more or less—be just a random start up. It doesn't have to be a random start up either.
 

cloudfair

Guy
Joined
Jan 6, 2025
Messages
71
Reaction score
281
Awards
41
I'm just going to come out with the fact that I do this for a living, admittedly among other things.
That is really really hard to believe
At this point I may as well stop reading because you don't seem to understand what "optimisation" means.
⠀⠀From Merriam-Webster:
Or perhaps what this means in the context of a search engine is unclear to you. In the context of search engines, optimisation means achieving the highest possible position in the search results.
I said SEO spam isnt the cause of bad search results...
In the context of this discussion yes. Google does not define the act nor the methods of search engine optimisation. Google kindly asks you to optimise in certain ways. This is because again and again Google finds itself fighting with people abusing their search engine to optimise the search engine result position in ways that make Google worse as a product.
You are very confused. Of course SEO does not literally refer to google, but in practice, SEO is whatever gets you the best ranking on google. Its funny you paint google like a site admin struggling against spammers and dosers, when in reality the search engine is working exactly as intended.
Yes, and found it an utterly pseud take.
Sure. I bet the company that's been refining its ranking algorithm for decades with unlimited compute and manpower is just totally overwhelmed by these genius SEO spammers. Just cant figure it out! Surely not by design for profit
True, but they make the most money by having the most quality search results. If you're interested in the money, I suggest you take a look on Alphabet Q1 2025 financials. Maybe you're interested in Alphabet's yearly report 2024.
⠀ ⠀ Google Ads on LLM spam sites are not so large of a source for income that it justifies butchering their ~equal (to their whole other ads network) marketing platform that is the search engine.
If they made money from having the most quality search results, then the search results would be good. They make 0 dollars from quality of search results but 200+ billion dollars from ads. So i think they probably care more about the ads, right?

Try to put these two things together and reach a conlusion. The search results are shitty and filled with ads and llm spam which are also filled with ads, and google is making more money than ever. Perhaps these are related.

False. Spending on Google Search ads improves performance of search engine
because you get more traffic to your website. ⠀⠀Google ads on your page worsens search engine position in a similar way as having Google's Tag Manager running on your website: more JavaScript to run.
Yes... buying google search ads improves your ranking thus the high ranked sites contain ads... glad you are starting to understand. And the second sentence is absolutely false. Google makes money when people click on the ads your site so they reward people who put google ads on their site.

Do you not see that everything you believe about how the algorithm and seo works directly contradicts what we actually observe with google? We see all the top results are filled with ads and llm generated, and your conclusion is that ads and llms hurt page rank?
Showing ads is not the only way to make money on the web and showing Google ads is in fact a comparatively bad way to make money on the web. I don't show any Google ads and none of my clients that are involved with me on the matter show any Google ads.
⠀⠀I do advertise on Google ads and much more than that I advertise on the search.
Cool
They have no use in storing the full page data. They have no use in storing the full page content either.
True, but they don't need the full page data or content to do this.
Of course they need the content thats what is used to train
I can't know of the bots scraping Agora because I'm not the webmaster. Neither can you.
Screenshot 2025-06-19 at 1.17.44 AM.png

Every company used to—more or less—be just a random start up. It doesn't have to be a random start up either.
Inspirational ending to a pretty retarded rant
 

Regal

Well-Known Traveler
Joined
Nov 20, 2022
Messages
399
Reaction score
1,575
Awards
132
The paying for search is depressing. It is, in effect, a sign of the ongoing death of the utopian ideals of the early web.

It says a lot about the costs of these services that only after reaching 50k paying subscribers Kagi is breaking even. The free internet may never have been sustainable. Not once you reach a certain level of users anyway.

As an aside, Kagi recently announced that they are going to release Kagi Search for free with some pretty reasonable usage limits. Kagi is made by people like us and they consistently seem to be trying to reach the utopian web you're talking about as much as they can.

(ps - Kagi if you ever read this I expect you to declare me a brand ambassador for shilling you and remove my subscription fee :agpepsi: )
 

polar_coyote

Traveler
Joined
Nov 8, 2024
Messages
120
Reaction score
262
Awards
49
I've been using DDG and Startpage as search engines lately (I end up using google when those fail me). Not sure what will happen to Startpage later, as it seems it's been bought by "big corpo", but for now, it's alright. I like it better than DDG in that it finds/understands non-english searches a bit better for some reason. DDG is kinda ass at the moment for non-english. Struggles to find most basic stuff sometimes.
But in some ways, at least both of them remind me of search engines before.

I remember way back, it was like "super important" that engines should find your seach faster than any other engine. If it used a fraction of a second more than another engine, it was shitty apparently.
Back then I used alltheweb or something, cause my dad insisted it was the best. Oh man there's a wiki page for it :3 https://en.wikipedia.org/wiki/AlltheWeb

Google on mobile was completely broken for me almost all of 2024. I used google to look up random gaming-things while gaming, and every time the first page was a garbled mess of random .pdf file links, "books.google" shit with the most RANDOM inchoherent text that was not linked to my query, etc. Just trash every time.
 

how to drain your dragon

Posting Inquisitor
Joined
May 28, 2022
Messages
1,223
Reaction score
4,164
Awards
255
I want my services to be good :(((

I also don't want to pay a red cent for them ((((((

pick one, you can't have both. if you can't afford it, stop being poor.
 
Virtual Cafe Awards

polar_coyote

Traveler
Joined
Nov 8, 2024
Messages
120
Reaction score
262
Awards
49
Does anyone know of a search engine where the before:0000 thing works anymore? I'm trying to find traces of something online, but DDG, google +++ keep coming in with 0 results. It used to give at least something or other, but now it just turns up with nada every time.

I used to be able to search for email too, in " " to see if emails were listed somewhere, and that too seems to be giving 0 results.

I'm also wondering if there is a way to actually search in the wayback machine? The site searching there either sucks ass or I cant understand it. I wish to be able to search their archives the same way one would use a search engine on the "live" web. (as in, with quotation marks, "before:0000", search for emails, text, names, etc.
 

Waninem

30 Year-Old Boomer
Joined
Aug 15, 2024
Messages
511
Reaction score
2,700
Awards
203
Thank you all for not using the word enshittification in this thread. I always see it pop up in discussions of this topic and I always cringe.
If I may ask, what's wrong with it? It's overused a bit, but it is a handy way to describe what's generally happening with large services.
 
Virtual Cafe Awards

K0WLOON

Well-Known Traveler
Joined
Feb 3, 2020
Messages
381
Reaction score
876
Awards
95
Website
kowl00n.neocities.org
If I may ask, what's wrong with it? It's overused a bit, but it is a handy way to describe what's generally happening with large services.
one isn't speaking for @sheo-chan, but this one's problem with the concept is that it seems to implicitly assume that the thing is emergent rather than the plan of the ones the entire time to cultivate a userbase from which to extract value. like some kind of crop of unsuspecting clodhoppers. A field of fools to fleece. a vineyard of virgin patsies. A garden of gormless rubes.

As if a quadspillion dollar for-profit enterprise accidentally discovered how to make money
:JahySmug:costanzayeahrightsmirk:LaughHard:
 
Virtual Cafe Awards

№56

Self-Hating Bureaucrat
Moderator
Gold
Joined
Mar 27, 2022
Messages
1,312
Reaction score
9,449
Awards
271
Website
no56.neocities.org
If I may ask, what's wrong with it?
I don't like how the best term we have to talk about online services sacrificing efficiency/user experience for the sake of profit also has to sound so stupid and meaningless. Imagine if you were totally ignorant of the issue and someone told you Google had become "enshittified." There's a very specific problem being referred to, the kind of thing lots of ordinary people have to deal with every day without really understanding what's going on, and the only real way to summarize what's happening is to say "Google is poopoo stinky now." That's not going to help my mom when she asks me why her phone keeps getting worse.
 
Virtual Cafe Awards