literature survey + master thesis: G-Rank learn-to-rank

[synctext]

Direction changed, txt will be updated soon.

Old stuff:

• 1997: The Internet: A Future Tragedy of the Commons?
• Internet Security And The Tragedy Of The Commons
• The Tragedy of the Commons: Twenty-two years later
• https://www.theatlantic.com/technology/archive/2012/05/the-tragedy-of-the-internet-commons/257290/
• Axiomatic principles: strongest possible reputation system #31 (comment)
• sybil-proof accounting of contributions to the commons .PDF
• Lots of scholar.google.com literature results
• 10 ECTS: https://studiegids.tudelft.nl/a101_displayCourse.do?course_id=51140

[awrgold]

• http://blogs.oii.ox.ac.uk/floridi/wp-content/uploads/sites/67/2014/05/ttotdc.pdf

• https://www.theatlantic.com/technology/archive/2015/06/the-tragedy-of-the-digital-commons/395129/

• https://www.hbs.edu/faculty/Publication%20Files/19-060_8c8c0f68-fecf-4aa7-8084-ee3583fa557a.pdf

tragedy2.pdf

[devos50]

Note that the tragedy of the commons is even present in blockchain systems (as a consequence of full state replication).

[awrgold]

New Literature Survey:

Autonomous (decentralized?) trading bot:

• (Possible) Features:

  • Online learning capable (?)
  • Decentralized deployment such that multiple copies of agents can learn with local information and compared to one another, i.e. federated learning(?)
  • Order flow vs. strategic indicator trading strategies
  • Generic enough to work for various asset pairs (potentially digital Euro markets??)
  • Autonomous portfolio (re)allocation

• Overleaf project:

  • https://www.overleaf.com/read/cdbnvbrnqsrr

• Starting literature:

  • https://dennybritz.com/blog/ai-trading/

• Seedboxes and decentralized storage and bandwidth allocation:

  • Is it more interesting than a decentralized trading bot?

[synctext]

True autonomous AI has full freedom to act, own and profit. This literature survey is focused on ...

• dataset privacy (multiparty-computation) not suitable, tech push
• robot collective learning (robo language between agents, send Q-table updates)
• agents which are really autonomous
• operate on a market and make money, https://github.com/Tribler/trustchain-superapp/blob/master/README.md#ai-trading-bot
• focussed: vpn, seedbox, and/or cybercurrency trading bot on decentral market
• Autonomous actor: Q-learning, local wallet with cybercurrency, and trading infrastructure: these are 3 primitives that have never been merged with true autonomy.
• Collectivity of autonomous entities: working together, Q-learning together, local wallet with cybercurrency, and trading infrastructure: these are 4 primitives that have never been merged with true autonomy.
  Related: "Collective Adaptive Systems: Challenges Beyond Evolvability"
  

Links

• $1 trillion savings in bank&insurance sector by replacing workers with AI, popular macro overview of AI&money
• Numerai, https://www.wired.com/2016/12/7500-faceless-coders-paid-bitcoin-built-hedge-funds-brain/
• The Rise of the Artificially Intelligent Hedge Fund

[awrgold]

Next step:

~10 sentences or so, narrowing the scope down to 3-4 directions to go in regarding the survey and prototype. (Next week)

[synctext]

This is the recent breakthrough in reinforcement learning algorithms (e.g. Asynchronous Proximal Policy Optimization (APPO) algorithm). Entire thesis goal/Survey-prototype could be to get this into the superapp AI bot, get a Tweakers post, deployment to a few hundred phones, measure, plot in thesis, and should scale to millions. Each smartphone has fixed 5-ish GByte slice of a huge dataset of historical market data, making each bot unique. Creates a very concrete focus: apply this ML technique to autonomous trading bots. Specifically, smarter bot trading this recent work on our decentral market; see here. Fancy results. Done. (only fundamentally re-work this technique by distributing the shared memory and decentralising the learner to offer unbounded scalability. Simple. /sarcasm)

Key research question: Can we map the “Sample Factory” (a high-throughput training system optimized for a single-machine setting) architecture to the Autonomous trading bot setting?

Website: https://sites.google.com/view/sample-factory
Article: https://arxiv.org/pdf/2006.11751
Press: https://spectrum.ieee.org/tech-talk/artificial-intelligence/machine-learning/powerful-ai-can-now-be-trained-on-a-single-computer
Code: https://github.com/alex-petrenko/sample-factory
Demo:

[awrgold]

Hey Johan, This looks interesting at first glance. I will start reading up on this before our meeting tomorrow. I just got back from a super spontaneous short holiday with some friends, so the last 5 days I have been away. I did have some great discussions with my friends though on this vacation (we all did our data science Bachelor together) so I am thinking I'm narrowing down the focus. Reinforcement learning is obviously the Mecca of "proper" AI as the theoretical ceiling on its potential is astronomical compared to (un)supervised learning, but of course is far more complex than simpler techniques. I will share more thoughts tomorrow. Current focus: adversarial training for trading bots, GANs, generative modelling, autonomous/online learning, q-policies, independent modelling, etc. See you then, Andrew

…

On Sat, Jul 18, 2020 at 11:03 AM Johan Pouwelse ***@***.***> wrote: This <https://arxiv.org/pdf/2006.11751> is the recent breakthrough in reinforcement learning algorithms (e.g. Asynchronous Proximal Policy Optimization (APPO) algorithm). Entire thesis goal/Survey-prototype could be to get this into the superapp AI bot <https://github.com/Tribler/trustchain-superapp/blob/master/README.md#ai-trading-bot>, get a Tweakers post, deployment to a few hundred phones, measure, plot in thesis, and should scale to millions. Each smartphone has fixed 5-ish GByte slice of a huge dataset of historical market data, making each bot unique. Creates a very concrete focus: apply this ML technique to autonomous trading bots. Fancy results. Done. (only fundamentally re-work this technique by distributing the shared memory and decentralising the learner to offer unbounded scalability. Simple. /sarcasm) Key research question: Can we map the “Sample Factory” (a high-throughput training system optimized for a single-machine setting) architecture to the Autonomous trading bot setting? [image: image] <https://user-images.githubusercontent.com/325224/87848851-1c3bc800-c8e4-11ea-8954-fe7a3521a423.png> Website: https://sites.google.com/view/sample-factory Article: https://arxiv.org/pdf/2006.11751 Press: https://spectrum.ieee.org/tech-talk/artificial-intelligence/machine-learning/powerful-ai-can-now-be-trained-on-a-single-computer Code: https://github.com/alex-petrenko/sample-factory Demo: <https://github.com/alex-petrenko/sample-factory/blob/master/gifs/battle.gif?raw=true> <https://github.com/alex-petrenko/sample-factory/blob/master/gifs/duel.gif?raw=true> — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub <#5313 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADFXMZWJLFO4ZUJE3FUUZ4LR4FQNDANCNFSM4M3KKCZQ> .

[synctext]

As a first survey step: please post here for Monday 24Aug: .PDF file in IEEE 2-column format with 20 citations of papers and first draft text in sections. Plus 1 giant table with all papers, per year, expanding something like:

Ideally add 1 graph of performance of experimental part. Idea: make a bandwidth trading bot ecosystem emulator with self-play. So 10 bots which do stuff and can be used to compare algorithmic complexity, intelligence + resilience against attack.

[synctext]

Can you find more in this:
https://papers.nips.cc/paper/5656-hidden-technical-debt-in-machine-learning-systems.pdf ?
"Hardest part of AI is not AI". You agree?

[synctext]

5 x 5ects courses left. plus lit survey. Proposed direction:

• self-sustaining AI
• please prepare for master thesis around a live system!
• trading pair: financial market or Tribler storage market
• reinforcement learning (there is better stuff, but this is mature stuff)
• data ownership issue is more of interest to the "policy management facility..."