Onboarding

[iancooper]

Hi @omelianlevkovyc,

Let's me know if you can see this, and the we can help you get started. I've attached a screen shot of our 'help wanted' section.

As an alternative, we need to work on our docs, and I am currently reviewing our samples. I am working on some WebAPI examples. I have one done for EF Core and I am working on a Dapper version of it. To cover of the most popular .NET data access libs we could do with another example for WebAPI and Marten.

The advantage of that would be that you could learn how to use Brighter, if you don't already know, and you can use the existing WebAPI and EFCore project to provide some ideas about how to write it.

Thoughts?

Ian

[omelianlevkovych]

Hi there everyone 👍
I agree that It'll probably be better to start with documentation so I can get onboard and have some look on the Brighter.

[omelianlevkovych]

@iancooper got it now. Adding new sample seems achievable.
I will start with getting WebAPI and EFCore samples up and running, as you already suggested.
By the way, is experience with Marten required? And I assume that by Marten you mean this: https://martendb.io/, am I correct here?

[iancooper]

No experience needed (I have none). Hopefully, useful learning for you. As I say, skip the outbox on v1, or use in-memory. Then we can write the integration with outbox with you.

[omelianlevkovych]

@iancooper help needed. Seems I am trying to do something completely wrong here.
Is it possible that we miss GreetingsAdapters directory in the WebAPI_EFCore? I can't run docker compose unfortunatelly.
ERROR: build path C:\personal\oss\samples\WebAPI_EFCore\GreetingsAdapters either does not exist, is not accessible, or is not a valid URL.

What is the right way to build the samples? Should I use build.sh script, or it is okay to build it from VS?
By the way, when trying to build the solution in Visual Studio I am getting the following error message:

Severity	Code	Description	Project	File	Line	Suppression State
Error	NU1104	Unable to find project '.\Paramore.Brighter.ServiceActivator.Extensions.HealthChecks\Paramore.Brighter.ServiceActivator.Extensions.HealthChecks.csproj'. Check that the project reference is valid and that the project file exists.	GreetingsWorker	.\samples\ASBTaskQueue\GreetingsWorker\GreetingsWorker.csproj	1	

However, after unloading that example project, I was able to build succesfully and also run the WebAPI EFCore example (not all endpoints are working but I assume it is due to not having RMQ up and running).

And the last question, sorry if it is too silly. How can I run tests for WebAPI EFCore example project. I see that we have test.http there. Any hints?
Thanks in advance :)

[iancooper]

So, the project is not built to be run in docker compose (the issue you are seeing is probably to do with a docker compose file needing to be at the root) but stand-alone. We tend to use docker compose to provide RMQ or MySQL. There are docker compose files in the root of the Brighter project that you can use for that.

[iancooper]

In principle it should build and run from the command line, but I use Rider and tend to run it from the launchsettings.json file, which will give me the correct environment variables for dev, production etc https://github.com/BrighterCommand/Brighter/blob/master/samples/WebAPI_EFCore/GreetingsWeb/Properties/launchSettings.json That chooses between Sqlite and MySQL as well, you will want a MySQL instance (hence the docker files in the root of Brighter) if you choose production.

[iancooper]

BTW if you don't know RMQ, localhost:15672/ or localhost:55672/ will give you the management console (usr:guest pwd:guest) where you can see what is happening, which really helps

[iancooper]

RMQ will drop a message, if it has no consumer. So you have to spin up the consumer - before you send messages, or they will be lost. Doing a POST of a greeting to a user should send a message

[iancooper]

Finally, yes, the tests.http file will work with Rider or RESTClient in Code to let you run tests against the WebAPI that should then drive messaging

[iancooper]

Let me know how it all goes!!

[omelianlevkovych]

Thanks a lot. Will try it out 👍

[omelianlevkovych]

Short update. Currently I am a bit stack with setting up RMQ.
It seems like I am doing something wrong when trying to connect publisher (GreetingsWeb) to my RMQ instance.
Here is the general message: RabbitMQ.Client.Exceptions.BrokerUnreachableException: 'None of the specified endpoints were reachable'.
If more detailed logs are needed, please let me know.
Hoping to manage this on my own, but I decided that this worth sharing, so I won't spend too much on something silly and can invest that time a bit smarter.

Additional information (hoping relevant):

• RMQ Management works fine on http://localhost:15672/#
• rabbitmq-diagnostics status shows (in docker container):

Totals

Connection count: 0
Queue count: 1
Virtual host count: 1

Listeners

Interface: [::], port: 25672, protocol: clustering, purpose: inter-node and CLI tool communication
Interface: [::], port: 5672, protocol: amqp, purpose: AMQP 0-9-1 and AMQP 1.0
Interface: [::], port: 15672, protocol: http, purpose: HTTP API

• I haven't changed any source code, epsecially configuration in Startup yet.
• I was able to run Consumer (SalutationAnalytics), created a queue.
  
• I do use the same creds: guest; guest
• my docker-compose.yml:

version: '3'

services:
  mysql:
    hostname: greetings_db
    image: mysql
    ports:
      - "3306:3306"
    security_opt:
      - seccomp:unconfined
    volumes:
      - my-db:/var/lib/mysql
    environment:
      MYSQL_ROOT_PASSWORD: "root"
    healthcheck:
      test: mysqladmin ping -h localhost -p$$MYSQL_ROOT_PASSWORD && test '0' -eq $$(ps aux | awk '{print $$11}' | grep -c -e '^mysql$$')
  rabbitmq:
    image: brightercommand/rabbitmq:3.8-management-delay
    ports:
      - "5672:5672"
      - "15672:15672"
    volumes:
      - rabbitmq-home:/var/lib/rabbitmq
volumes:
    rabbitmq-home:
      driver: local
    my-db:
      driver: local

I am planning to read more RMQ docs tomorrow as well; especially : rabbit-mq-networking-troubleshooting.
It anyway will be very useful in the nearest future, and I am afraid that I am missing something very basic here.

[omelianlevkovych]

Finally, I have noticed that for production we do not use localhost, changed AMPQ uri to amqp://guest:guest@localhost:5672 in the Startup and now I can see the connection in RMQ Managment.
Also RMQ now process messages:

I am also able to send greetings over postman.

Now I am planning to spin up the consumer and debug the message receiving process; and than I will move to see whether tests are working.

[omelianlevkovych]

I have added phpmyadmin and connected to the mysql instance hostname=greetings_db, pass&username=root
I can see in the Salutations table the sended message:

However, the Inbox table in the Salutations db is empty. Also I am not quite sure what I should see in the RMQ Managment:

Shortly, to make it works (publisher-rmq-consumer) I have changed host to localhost in production; not sure it is the right approach.
Now I will try to make tests to pass.

[iancooper]

Hi, sounds like you have made some progress. Yes, checking the AMQP connection string is always a good idea, as you need to make sure it points to the instance you are using for testing (in this case localhost).

[iancooper]

You would expect to see the following:

• Message in Outbox for the outgoing message
• A historical record of the message via message rates
  • If you want to, you can stop the consumer, run the producer, see the message sitting in the queue, and then run the consumer to eat it; this lets you see the asynchronous nature of the workflow
• The inbox should have a copy, though you need to make sure that the inbox table exists in the db (sqlite or mysql depending on what you are choosing)

One thing to watch for with sqlite - its read-only for subsequent runs and fails, as only the creating process has rights to write by default, unless you explicitly grant those rights. It's a PITA for multiple-runs. If that is the case, just ensure you are using your MySql instance for the inbox too.

If you turn diagnostics up enough, you should get error messages about failed writes and why.

[iancooper]

Let me know if that helps at all.

[iancooper]

But fantastic to see that you have gotten so far.

[omelianlevkovych]

@iancooper yea, you are very helpful, I appreciate it a lot.
Also hoping I am not very annoying 😅

• I do see messages in the Outpox table (in Greetings db); they also are getting dispatched
• I have stopped consumer and can now see that messages are staying in the queue. Nice.
  Unfortunatelly, despite having the Inbox and Outbox tables in the Salutations db, I still missing data there. Only the Salutations table is getting populated after consumer get messages from the queue.
  
  I am using Production launch settings and MySQL therefore, any ideas why my Inbox table is empty?

However, everything else seems to work. Lets see whether I can figure out how to run tests now. 😉

[omelianlevkovych]

Oh okay, so tests.http its more like postman collection rather than some integrational tests.
@iancooper In case I will figure out why I don't have anything in Inbox table, what is our next step?
updated: I have tested with the endpoints in tests.http via VS Code extension, all seems to work.
The last question is whether I should have any data in the Inbox table (Salutations db).

[iancooper]

You are not annoying at all. Happy to onboard new people, and I learn a lot from figuring out how easy it is for new folks to get end-to-end.

What would be really valuable is if you can think of how we could update the readme.md for this sample, to make it easier. What did you not understand? Can you submit a PR with an update for the sample's readme.md for that? What you don't now is what most won't know, and is hard for us deep in the project to understand.

If the http calls all work there ought to be data in the inbox. If not, we might have a bug. There is an explicit attribute on the handler for the Inbox, but it is commented out IIRC. Try commenting it in, and see if that makes it work. In that case there may be an issue with the"auto" Inbox

[iancooper]

Once you can figure out how to get Brighter working, the steps would be:

• PR the sample readme if you can, to record what you had to discover
• Figure out if the Inbox has a bug

[iancooper]

After that, it's either,

• We figure out what is wrong with the Inbox, or the sample
• We look at how we could replace EF Core with Marten (https://martendb.io/introduction.html) for another example
• • This will have a simple part (get it working without Outbox support) and a hard part (Outbox support for Marten)

[iancooper]

PS Yes, tests.http is just a lightweight version of postman effectively

[omelianlevkovych]

@iancooper I uncommented the attribute and inbox works now 😎
I would appreciate to make the PR for this particular example update.
However, I am not sure whether I have to understand how all that works, because honestly I don't.
So I see two possible ways of doing it, depends on what we need.

• I can update readme with things I struggled when spinning it up (publisher, rmq&mysql, consumer). And than learn more by implementing Marten example (also writing doc for that, and probably I can than add more things I learned into doc for EFCore_MySQL).
• Or I should basically spend more time figuring out what exactly Brighter does, all that clean architecture benefits it brings, etc.

[omelianlevkovych]

jfi, I have created two issues

• one for the potential bug with Inbox
• another one is for the readme update for EFCore_WebApi example; It seems to be the right way before creating PR; in case I understand Contributing documentation correctly. Hoping to create PR soon.

also created the PR for the doc update

[omelianlevkovych]

So if we have docker compose files in the root (so we can spin up containers like RMQ), why do we need another one in the example folder?

Also asked this in the PR, maybe we should continue discussion about docker compose there, idk

[preardon]

@omelianlevkovych let me know if I can be of any assistance

[omelianlevkovych]

@preardon Sure.
Could you please have a look to my PR #2054. For some reason CI for AWS has failed and I do not have permissons to rerun it.
Also I am not sure whether I should link the issues I created for the doc with the PR.
Thanks a lot in advance.

[preardon]

@omelianlevkovych This is a problem with our AWS Test being flaky, I will have a look at the PR and add any appropriate comments

[omelianlevkovych]

@preardon Okay, thanks for your comment, it motivates a lot.
By the way I have pushed hopefully the last updates to this PR.

I will start with Marten example today, so probably will have some questions sooner or later.

[iancooper]

I merged the PR for you.

[iancooper]

Sorry it took so long, busy day.

PS @preardon is a good source of help if I am not around (along with @holytshirt and @jonnyollifflee)

[omelianlevkovych]

@iancooper , @preardon Hi guys, do we have some documentation about how to properly configure the Brighter in Startup?
I am currently having an issue with DI that It doesn't see Handler for the Command.
System.ArgumentException: No command handler was found for the typeof command GreetingsPorts.Requests.AddPerson - a command should have exactly one handler.
I assume this extensions should figure out my DI setup AutoFromAssemblies.
I have pushed the code to webapi_marten_example branch

• Do I have to use Message Broker even though I am using CQRS (basically the publisher API; for instance, to create a Person)
• Does Brighter support Outbox pattern under the hood, can I somehow setup distributed transacrtion? And for my case UseInMemoryOutbox should be just enough so far?

[omelianlevkovych]

@preardon Hah, sorry for that.
I can update to dotnet 6 version, rebuild and commit. It than should be easier for you I guess. Should I do it now?

[preardon]

Got it working

[preardon]

There are a few things that I needed to do here

Firstly it appears that it wasn't running from the correct dll.

I first tried being a little more specific
.AsyncHandlersFromAssemblies(typeof(AddPersonHandlerAsync).Assembly)

This picked up your other handler that had an unmet dependency so I added

services.AddSingleton<GreetingsEntityGateway>();

This then started complaining about too many handlers, so I removed the AutoFromAssemblies and it start working.

to confirm my suspicion of Lazy Loading I removed the AsyncHandlersFromAssembly as above and added

var tst = new AddPerson("Paul");

and it continued to work!!

tldr; lazy loading stopped Auto from working, if you were to load a service earlier then the dll should be loaded all you need to add is
services.AddSingleton<GreetingsEntityGateway>();

I am not sure of your knowledge level, please let me know if you would like any more clarification than above

[omelianlevkovych]

Thanks a lot, that is very clear explanation. Obviously it also works for me now 👍

[preardon]

Excellent, let me know if I can be of any more help

[omelianlevkovych]

Do you guys aware about any sqlite similar document database, which I can use for the Development environment for the Marten example.

Or for now its ok to use postgresql for both.
marten doc about dev and prod envs

[omelianlevkovych]

And also I am wondering whether I should create a PR.
I have added the Greetings + Marten part only for now.
It still lacks the Salutations (consumer), RMQ config and outbox pattern, update to net6.

It would be nice to get some critique earlier on, before I made something really stupid and than had to rollback it all.
On the other hand, I don't want to bother much.

[jonnyolliff-lee]

@omelianlevkovych

Do you guys aware about any sqlite similar document database, which I can use for the Development environment for the Marten example.

I'm not up on Marten, but I'll try and do some reading and see. Someone else might have a better idea.

And also I am wondering whether I should create a PR.

it's worth creating a "Draft" PR if you want to get some early thoughts, but you're not completely ready.

[omelianlevkovych]

@jonnyolliff-lee thanks a lot!

[omelianlevkovych]

@iancooper Hey, Is the example for Marten still a thing? I just can not find the issue anymore.
I would like to get back to that and finalize the PR (actually I committed from the wrong git user and will probably recreate the PR).

[iancooper]

@omelianlevkovych Your PR still exists: #2076 but you may need to merge with master to pick up recent changes. Welcome back!