Hash versus giant string?

[johnseekins-pathccm]

Someone has written an MIT-licensed Redis brain that uses hget/hset commands for adding removing data from the brain. Is it worth exploring this more broadly for the main Redis brain? I think it would make a horizontally scalable Hubot backed by Redis possible?

[joeyguerra]

Yes. I think so because this one just stores everything in a single JSON document and saves that document to Redis. That large granularity has scaling and usage limits.

@johnseekins-pathccm do you have a current use case? Or were you just thinking about possible use cases for the future?

[johnseekins-pathccm]

I mean...I'd like our deploys of Hubot to not require downtime. Which using this brain does require. Also, I think it would address issues like #39 .

[joeyguerra]

Can you elaborate on what's causing the downtime?

[johnseekins-pathccm]

My understanding is we can only have one instance of Hubot running at a time (because the brain reads/writes a string, which can overwrite values between two instances). Thus, we tear down the old instance before starting the new one.

[joeyguerra]

I'm speculating here ... I think the problem with > 1 instances of Hubot is that in the "save to Redis" scenario, the last write wins. Which could result in data loss, multiple instances writing at the same time. Both of the hubot-redis implementations have this problem. I think it's a design problem with the Hubot Brain. It assumes that there's just 1 instance of Hubot and that all data is kept in memory. I believe that was the intent.

[johnseekins-pathccm]

In that case, I guess we leave this be as there's no real difference. Other than some scenarios have slightly improved read/write speed. Which isn't enough of an argument.

[joeyguerra]

I'm fine with keeping this issue open just in case others want to chime in. I'm interested in redesigning the Brain to accommodate multiple Hubot instances. And I'd also like to redesign the Hubot-redis design to not use JSON, but leverage some of Redis' data structures instead.

[johnseekins-pathccm]

I'd love to see Hubot actually able to run more than one instance. If it's just the internal API for the brain causing the issues, that seems solvable. I'm no javascript guy, but if I can help, I'm happy to try.

[joeyguerra]

I took a stab at a design to scale Hubot instances. hubot-redis-inbox.

I'll take a stab at redesigning this codebase to move the brain into Redis instead of holding it all in the Hubot process.

[joeyguerra]

I forked this repo into the hubot-friends org so I can redesign it to store data in a more ganular way in Redis.