Akka.Streams.Kafka v1.5.40.1

1.5.40.1 March 24th 2025

• Fixed another major N+1 bug when committing offsets to Kafka: Committing: fix N+1 issues in CommittableOffsetBatch

Changes:

• 99f3dab added v1.5.40.1 release notes (#488)
• fd66c8d Committing: fix N+1 issues in CommittableOffsetBatch (#487)

This list of changes was auto generated.

Akka.Streams.Kafka v1.5.40

1.5.40 March 24th 2025

• Major bug fix: Committers: fixed n+1 errors with offset committing
• Upgraded to Akka.NET v1.5.40

Changes:

• 45433a8 Added v1.5.40 release notes (#486)
• 0c80ff7 Upgraded to Akka.NET v1.5.40 (#485)
• 2a191f7 Committers: fixed n+1 errors with offset committing (#484)
• 3e12279 reformatted (#482)

This list of changes was auto generated.

Akka.Streams.Kafka v1.5.39

1.5.39 March 19th 2025

Akka.Streams.Kafka 1.5.39 represents a major improvement in stability and performance for Kafka stream processing, particularly for applications using manual partition assignment and rebalancing scenarios. This release includes critical fixes for partition management and introduces new performance tuning capabilities that give users more control over their Kafka consumer behavior.

Key Improvements

• Improved stability during partition rebalancing operations
• Enhanced manual partition assignment behavior
• Significantly more reliable and improved committing performance
• New performance tuning capabilities for consumer polling
• Better handling of partition revocation scenarios
• Improved memory efficiency through C# record types

Breaking Changes

• Manual partition assignment now uses IncrementalAssign instead of Assign - this prevents offset resets for users running ManualSubscriptions. If you're using manual partition assignment, you'll need to verify your offset management logic is compatible with incremental assignment behavior.
• Several internal types have been converted from classes to C# records for better performance and nullability support. This change should be transparent for most users as records are fully compatible with standard class usage patterns. The only potential impact would be if you're using inheritance on these types (which is not a recommended pattern for Akka.Streams.Kafka types).
• We removed some extension methods that should have never been made public in the first place.
• We made some changes to ICommittable interface and others.

Major Bug Fixes and Improvements

• Fixed critical issue: Exception inside SelectAsync with null cancellation cause
• Resolved: System.ArgumentException during rebalance operations
• Added performance tuning capability through ConsumerSettings.MaxPollRecords
• Improved partition management: filtering messages from revoked partitions
• Enhanced stability: filtering out buffered records from recently revoked partitions

Performance Data

For the PlainSource:

                                                                                     
BenchmarkDotNet v0.14.0, Pop!_OS 22.04 LTS                                           
13th Gen Intel Core i7-1360P, 1 CPU, 16 logical and 12 physical cores                
.NET SDK 9.0.100                                                                     
  [Host]  : .NET 9.0.0 (9.0.24.52809), X64 RyuJIT AVX2                               
  LongRun : .NET 9.0.0 (9.0.24.52809), X64 RyuJIT AVX2                               
                                                                                     
Job=LongRun  EvaluateOverhead=False  Concurrent=True                                 
Server=True  InvocationCount=1  IterationCount=10                                    
LaunchCount=3  RunStrategy=Monitoring  UnrollFactor=1                                
WarmupCount=3  Categories=MacroBenchmark,Consumer,Plain                              
                                                                                     

Method	PollBatchSize	Mean	Error	StdDev	msg/sec
ConsumeMessageAsync	500	41.29 μs	1.960 μs	2.933 μs	24,217.30

This is a ~2.5x improvement over what v1.5.38 was able to achieve.

Dependencies

• Upgraded to Akka.NET v1.5.39

1.5.39-beta2 March 14th 2025

• Upgraded to Akka.NET v1.5.39
• Resolved: Kafka Producer - Exception occured inside SelectAsync - Cancellation cause must not be null

1.5.39-beta1 March 13th 2025

v1.5.39 is a major update for Akka.Streams.Kafka

• Resolved: System.ArgumentException: Unexpected records polled potentially thrown during a rebalance
• Expose ConsumerSettings.MaxPollRecords available so users can performance-tune how many records to fetch during polling.
• Change Assign and AssignWithOffsets to use IncrementalAssign - prevents Offset resets for users running ManualSubscriptions
• Refactor SubSourceStageLogic; filter messages from revoked partitions in partitioned stream sources
• Filter out buffered records from recently revoked partitions
• Enable nullability

Changes:

• 1aeacf5 Updated README.md (#480)
• 7e8eeb4 Added v1.5.39 release notes (#479)
• f18fdd9 restructured our CommittableOffset benchmark (#478)
• da1498e Performance tuning consumer and committer settings (#472)
• 5666fbc Update CommitCollectorStageBenchmark.cs
• c256a88 added CommitCollectorStage benchmark (#477)
• e5ddd61 fix issues with CommittableSource benchmarks (#476)
• 125b25a Created CommittableOffsetBatchBenchmark (#474)
• aa94008 readjust all benchmarks to use just the default values (#473)
• 592faa6 Add specs for Committing stages (#471)

See More

• 08a6d5f KakfaConsumeActor: Stopping behavior and shutdown loggingi (#470)
• 199d0db Adding support for configurable commit collecting / draining (#458)
• 8ab0d82 make ProducerRecord a record (#469)
• 4e5ecb3 Migrated ConnectionCheckerSettings to use record types (#468)
• f079d35 Bump tests and example / benchmark runtime to .NET 9 (#467)
• 18c525e restructure and expand benchmarks (#466)
• 2bd3733 fix IControl and ensure clean draining / termination of Akka.Streams consumers (#464)
• d60c4f7 quality: exclude obsolete and non-essential code from coverage reports (#463)
• d020b18 quality: generate and publish CRAP scores to AzDo (#462)
• ced3532 fixed PowerShell (#461)
• 090e127 Update nightly-builds.yaml for Azure Pipelines
• b193311 remove FAKE, simplify build system (#460)
• c82461d Added v1.5.39-beta2 release notes (#459)
• 0788a54 upgraded to Akka.NET v1.5.39 (#457)
• 1c72293 Added v1.5.39-beta1 release notes (#456)
• 359aa54 Resolve ArgumentException upon partition re-assignment (#454) [ #415 ]
• 71a6606 Change Assign and AssignWithOffsets to use IncrementalAssign (#455)
• 97fb44a Add RebalanceExtTests for testing partitioned message loss on rebalance (#453)
• aead3e4 Refactor SubSourceStageLogic; filter messages from revoked partitions in partitioned stream sources (#452)
• 215a232 Rewrite RebalanceIntegrationTests; filter out buffered records from recently revoked partitions (#450)
• 573437d Enable nullability (#449)
• a39e933 [WIP] fix nullability / build warnings (#448)

This list of changes was auto generated.

Akka.Streams.Kafka v1.5.39-beta2

1.5.39-beta2 March 14th 2025

• Upgraded to Akka.NET v1.5.39
• Resolved: Kafka Producer - Exception occured inside SelectAsync - Cancellation cause must not be null

1.5.39-beta1 March 13th 2025

v1.5.39 is a major update for Akka.Streams.Kafka

• Resolved: System.ArgumentException: Unexpected records polled potentially thrown during a rebalance
• Expose ConsumerSettings.MaxPollRecords available so users can performance-tune how many records to fetch during polling.
• Change Assign and AssignWithOffsets to use IncrementalAssign - prevents Offset resets for users running ManualSubscriptions
• Refactor SubSourceStageLogic; filter messages from revoked partitions in partitioned stream sources
• Filter out buffered records from recently revoked partitions
• Enable nullability

Changes:

• c82461d Added v1.5.39-beta2 release notes (#459)
• 0788a54 upgraded to Akka.NET v1.5.39 (#457)
• 1c72293 Added v1.5.39-beta1 release notes (#456)
• 359aa54 Resolve ArgumentException upon partition re-assignment (#454) [ #415 ]
• 71a6606 Change Assign and AssignWithOffsets to use IncrementalAssign (#455)
• 97fb44a Add RebalanceExtTests for testing partitioned message loss on rebalance (#453)
• aead3e4 Refactor SubSourceStageLogic; filter messages from revoked partitions in partitioned stream sources (#452)
• 215a232 Rewrite RebalanceIntegrationTests; filter out buffered records from recently revoked partitions (#450)
• 573437d Enable nullability (#449)
• a39e933 [WIP] fix nullability / build warnings (#448)

This list of changes was auto generated.

Akka.Streams.Kafka v1.5.39-beta1

1.5.39-beta1 March 13th 2025

v1.5.39 is a major update for Akka.Streams.Kafka

• Resolved: System.ArgumentException: Unexpected records polled potentially thrown during a rebalance
• Expose ConsumerSettings.MaxPollRecords available so users can performance-tune how many records to fetch during polling.
• Change Assign and AssignWithOffsets to use IncrementalAssign - prevents Offset resets for users running ManualSubscriptions
• Refactor SubSourceStageLogic; filter messages from revoked partitions in partitioned stream sources
• Filter out buffered records from recently revoked partitions
• Enable nullability

Changes:

• 1c72293 Added v1.5.39-beta1 release notes (#456)
• 359aa54 Resolve ArgumentException upon partition re-assignment (#454) [ #415 ]
• 71a6606 Change Assign and AssignWithOffsets to use IncrementalAssign (#455)
• 97fb44a Add RebalanceExtTests for testing partitioned message loss on rebalance (#453)
• aead3e4 Refactor SubSourceStageLogic; filter messages from revoked partitions in partitioned stream sources (#452)
• 215a232 Rewrite RebalanceIntegrationTests; filter out buffered records from recently revoked partitions (#450)
• 573437d Enable nullability (#449)
• a39e933 [WIP] fix nullability / build warnings (#448)

This list of changes was auto generated.

Akka.Streams.Kafka v1.5.38

1.5.38 March 4th 2025

• Upgraded to Akka.NET v1.5.38
• Resolved: System.ArgumentException: Unexpected records polled potentially thrown during a rebalance

Changes:

• 994f40d Added v1.5.38 release notes (#445)
• 14819f4 Cleanup extra debug logging (#444)
• 36d5eaf [WIP] rebalance integration specs (#443)
• 7fcdeb3 Make partitions-per-topic configurable (#442)
• ef22965 added coverlet for code coverage (#441)
• b1181f5 KafkaConsumerActor: remove _partitionsToResume (#440)
• e7e6380 More doc fixes (#434)
• 3c8d6cc Refactor KafkaConsumerActor Scheduler to use Timers (#436)
• fb98615 Revert "Revert #418 (#431)" (#435)
• 73f788a Revert #418 (#431) [ #422, #424, #425, #427, #423, #428, #429, #430 ]

See More

• d623aba XML-DOC improvements (#433)
• 2d11235 Add CancellationToken support to MockCustomer (#430)
• 8e41f71 Make sure that restricted consumer is populated properly (#429)
• 02c002e Bump Confluent.Kafka to 2.8.0 (#428)

This list of changes was auto generated.

Akka.Streams.Kafka v1.5.37

1.5.37 February 6th 2025

• Upgraded to Akka.NET v1.5.37
• Refactor KafkaConsumerActor event handling
• Implement missing OnPartitionsLost event handler
• Introduce Partition extension to optimize LINQ
• Prevent null exceptions propagation from downstream stage

Breaking Change

There's a breaking public API inside IPartitionEventHandler interface. If you have a custom partition event handler, you will now need to implement the OnLost partition event handler delegate.

1.5.35 January 20th 2025

• Upgraded to Akka.NET v1.5.35
• Downgrade Confluent.Kafka from 2.4.0 to 2.2.0

1.5.33 December 26th 2024

• Upgraded to Akka.NET v1.5.33
• Fix TransactionalSourceLogic partition drain bug

1.5.30 October 3rd 2024

• Upgraded to Akka.NET v1.5.30
• Bump Confluent.Kafka to 2.4.0

1.5.29 October 1st 2024

Note

Deprecated

Deprecated due to Akka.NET 1.5.29 deprecation. Please use 1.5.30 instead.

• Upgraded to Akka.NET v1.5.29
• Bump Confluent.Kafka to 2.4.0

1.5.15 January 11 2024

• Upgraded to Akka.NET v1.5.15

1.5.13.1 October 4 2023

• Publish NuGet package symbol file

1.5.13 October 4 2023

• Upgraded to Akka.NET v1.5.13
• Bump Confluent.Kafka to 2.2.0

1.5.8 June 16 2023

• Upgraded to Akka.NET v1.5.8

1.5.0 March 02 2023

• Upgraded to Akka.NET v1.5.0

1.4.49 January 27 2023

Akka.Streams.Kafka version will now be in lock-step with its Akka.NET package version.

• Upgraded to Akka.NET v1.4.49
• Handle exceptions thrown from kafka message processing
• Bump Confluent.Kafka from 1.9.3 to 2.0.2

1.2.2 October 21 2022

• Upgraded to Akka.NET v1.4.45

Changes:

• d6b0a42 Update RELEASE_NOTES.md for 1.5.37 release (#423)
• 562212d Make sure that we prevent downstream to propagate null completion exception (#427)
• 36643b2 Cleanup linq by introducing Partition (#425)
• 6c895c5 Implement dual targeting for netstandard2.0 and net6.0 (#424)
• d2aa70f Bump AkkaVersion to 1.5.37 (#422)
• 92c4600 Refactor KafkaConsumerActor event handling from Mailbox to direct method call (#418)

This list of changes was auto generated.

Akka.Streams.Kafka v1.5.35

1.5.35 January 20th 2025

• Upgraded to Akka.NET v1.5.35
• Downgrade Confluent.Kafka from 2.4.0 to 2.2.0

Changes:

• fecaddb Update RELEASE_NOTES.mc for 1.5.35 release (#421)
• 3e62d3d Bump AkkaVersion to 1.5.35 (#419)
• 2f013d8 Downgrade Confluent.Kafka to 2.2.0 (#420)
• 4d94f9e Modernize unit test (#417)
• 3bf0948 fix typos in KafkaConsumerActor (#416)

This list of changes was auto generated.

Akka.Streams.Kafka v1.5.33

1.5.33 December 26th 2024

• Upgraded to Akka.NET v1.5.33
• Fix TransactionalSourceLogic partition drain bug

Changes:

• cabc41e Update RELEASE_NOTES.md for 1.5.33 (#412)
• 610cc4b Bump AkkaVersion from 1.5.30 to 1.5.33 (#411)
• 34a3eaf Fix possible TransactionalSourceLogic partition drain bug (#410)

This list of changes was auto generated.

Akka.Streams.Kafka v1.5.30

1.5.30 October 3rd 2024

• Upgraded to Akka.NET v1.5.30
• Bump Confluent.Kafka to 2.4.0

1.5.29 October 1st 2024

Note

Deprecated

Deprecated due to Akka.NET 1.5.29 deprecation. Please use 1.5.30 instead.

• Upgraded to Akka.NET v1.5.29
• Bump Confluent.Kafka to 2.4.0

1.5.15 January 11 2024

• Upgraded to Akka.NET v1.5.15

1.5.13.1 October 4 2023

• Publish NuGet package symbol file

1.5.13 October 4 2023

• Upgraded to Akka.NET v1.5.13
• Bump Confluent.Kafka to 2.2.0

1.5.8 June 16 2023

• Upgraded to Akka.NET v1.5.8

1.5.0 March 02 2023

• Upgraded to Akka.NET v1.5.0

1.4.49 January 27 2023

Akka.Streams.Kafka version will now be in lock-step with its Akka.NET package version.

• Upgraded to Akka.NET v1.4.49
• Handle exceptions thrown from kafka message processing
• Bump Confluent.Kafka from 1.9.3 to 2.0.2

1.2.2 October 21 2022

• Upgraded to Akka.NET v1.4.45

Changes:

• cf4b09c Update RELEASE_NOTES.md for 1.5.30 release (#409)

This list of changes was auto generated.