Feature Request: End-to-End Logging Including Intermediate Steps in crewAI

[Shivanandroy]

Hello crewA Team,

Thank you for this awesome project!

I was wondering if there is a possibility of enhancing the logging capabilities of the crewAI. Currently, I am able to log the output of tasks, but an integrated feature for end-to-end logging, capturing all intermediate steps, would significantly improve the utility and debugging capabilities of the package.

[greysonlalonde]

@joaomdmoura I like this - I think a well-implemented auditing feature is needed for prod environments. Options for writing to various data stores, etc

[joaomdmoura]

Indeed, log is a must, right now you can it with LangSmight and other but we should have an native option, marking as feature accepted

[JaDenis]

I agree, i'd like to send everything that is printed in console via WebSockets to a web client.
I can't make everyone i work with install VSCode and work with code, they need some kind of web app.

[mdfederici]

I also really need this feature to go to production using crewAI. I would like to store this information in a separate data store. Ideally I would be able to easily tell for each agent, each step in their thought process - what task they were trying to solve, what tool they used, what the input and output of the tool was, etc.

[x-technik]

This feature would be very nice to have.- Thanks for this awesome product.

[JasonCoBA]

Please implement this .
An option in the agent that yields or streams its thoughts, actions is required.

I’m trying to implement this with websocket myself, but it’s challenging and should be done lower down.

[sbhadana]

what are other frameworks other than langsmith supported?

[gadgethome]

what are other frameworks other than langsmith supported?

You can try langtrace, langfuse and AgentOps.

[sbhadana]

what are other frameworks other than langsmith supported?

You can try langtrace, langfuse and AgentOps.

langfuse is giving object not calleable error. Langsmith works out of the box.
Any workaround to get it by langfuse as well?

[gadgethome]

Maybe contact langfuse support and see if they have any known issues. thanks

[ashokgit]

Any open source tools for logging?

[nethi]

I ended up starting a discussion on similar topic here. Any
thoughts on supporting this ? Most agent frameworks are supporting some form of callbacks for intermediate steps etc. I have tried using step_callback and task_callback. It seems like these callbacks are post the action, so it is difficult to render them without any context. For example, I get a step_callback from an agent first and then the respective task callback after, with no context to stitch them together in the UI

#1389

[avishwa23]

Hi, @joaomdmoura new to crewai here, great framework first up....Now, I basically want to log the tool inputs and the outputs for my agent evaluation. What is the best way to do this currently? thanks.

[JaDenis]

I came to realization that reading intermediate generations is a waste of time.
the end results are very very mediocre too, no matter what LLM I use. I can get maybe a few good phrases, but never a solid document.

offtopic: alcohol and drugs are still the way to go for creative writing