[Proposal] Rewrite (or port) YaCy as serverless webapp with typescript and WASM #723
Description
As unrealistic as it may seem. After all, there is some support of JVM in WASM (could be sluggish though, but still a good start). Cheeprj literally has jvm as runtime in web-assembly (solr might be able to run in it).
Nowadays, P2P communication can be done right from browser using WebRTC. It would only need public nodes for peer discovery rather than communication. Let's say, peerjs library already provides discovery nodes.
This way whole YaCy can become a single webpage application hosted let's say here on github pages. Internal browser's IndexedDB can be used as a storage (it allows for gigabytes of data). Separate services would become separate webworkers communicating through let's say js-events or something high-level. Same architecture - different infrastructure.
Search engine right on GitHub webpage!
Another option is to start from scratch.
The main concern with yacy is DDoS security. If Yacy gets popular - it would be competing with commercial search engines, which means they would do their best to garbage yacy nodes with fake URLs.
The way Google Groups was ruining Usenet with spam
In my mostly unrelated P2P project (true oracles for bitcoin contracts):
https://dk14.github.io/mega-peers/docs/#/
I solved the DDoS issue by using:
- Bitcoin-like mempools as a data storage (my nodes store records in hashtable with an option for sharding, so can be DHT just not specialized to it yet).
- PoW as a way to protect from spam
In case of YaCy (as a protocol) - it would be minor upgrade of data structure. Just require PoW over a record (e.g. "reverse word to URL" record) and evict weak records from DHT-shard with PoW-difficulty ranking.
NO (!) need for anything cryptocurrency nonsense here. Presearch made a great mistake issuing a token (ERC is what killed Enigma SMPC, same fate awaits for Presearch).
I only use Btc as an example of hashcash - NO need for any incentives or rewards and such. User opens a webpage - user participates in the network.
WebRTC allows direct two-way communication for most clients nowadays (my mega-peers has an experimental trading app demonstrating that). So most nodes in "YaCy Web" would be "senior" nodes.
Third, a bit weird option. Since my "mega-peers" protocol has P2P and PoW already - can add ReverseWord message type to it, and create separate webcrawler service. This way if my (mega-peers) protocol gets adoption - such search engine would get adoption too.
I don't issue any stupid tokens - so it is still independent open-source protocol. A bit strange to add something unrelated, but since my protocol is aimed at P2P "government" among other things (mega-peers is: automated "human ID" issuance + attestations of real world events + financial contracts; it can do anything government can, except that now you'd be ordering those services directly P2P without man-in-the-middle). Thus, mega-peers would benefit from free P2P search engine as a way for humans to advertise and discover such services independently from google and such.