Skip to content

Release v0.37 #901

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 23 commits into
base: main
Choose a base branch
from
Draft

Release v0.37 #901

wants to merge 23 commits into from

Conversation

mhauru
Copy link
Member

@mhauru mhauru commented Apr 24, 2025

0.37 checklist

Accumulators

Accumulators broke Enzyme. This is a known issue and already reported upstream:

Remove context from evaluate!! and simplify submodel code

Replacing SamplingContext with InitContext

Fixes to AD testing

Shuffle code around

  • Break up contexts and possibly accumulator code

Possibly for this version or next minor version?


Not for this version (?)

Things that can be patch releases

@mhauru mhauru marked this pull request as draft April 24, 2025 16:47
Copy link

codecov bot commented Apr 24, 2025

Codecov Report

Attention: Patch coverage is 87.59232% with 84 lines in your changes missing coverage. Please review.

Project coverage is 82.67%. Comparing base (ce7c8b1) to head (2074657).

Files with missing lines Patch % Lines
src/simple_varinfo.jl 62.22% 17 Missing ⚠️
src/abstract_varinfo.jl 79.10% 14 Missing ⚠️
src/varinfo.jl 88.52% 14 Missing ⚠️
src/compiler.jl 53.84% 6 Missing ⚠️
src/logdensityfunction.jl 71.42% 6 Missing ⚠️
src/test_utils/ad.jl 66.66% 6 Missing ⚠️
src/threadsafe.jl 86.36% 6 Missing ⚠️
src/debug_utils.jl 85.71% 4 Missing ⚠️
src/extract_priors.jl 81.81% 4 Missing ⚠️
src/context_implementations.jl 93.93% 2 Missing ⚠️
... and 3 more
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #901      +/-   ##
==========================================
- Coverage   82.97%   82.67%   -0.31%     
==========================================
  Files          36       38       +2     
  Lines        3965     4022      +57     
==========================================
+ Hits         3290     3325      +35     
- Misses        675      697      +22     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@coveralls
Copy link

coveralls commented Apr 24, 2025

Pull Request Test Coverage Report for Build 16176308335

Details

  • 565 of 669 (84.45%) changed or added relevant lines in 27 files are covered.
  • 49 unchanged lines in 12 files lost coverage.
  • Overall coverage decreased (-0.2%) to 82.918%

Changes Missing Coverage Covered Lines Changed/Added Lines %
src/utils.jl 7 8 87.5%
src/accumulators.jl 27 29 93.1%
src/context_implementations.jl 31 33 93.94%
src/experimental.jl 0 2 0.0%
src/values_as_in_model.jl 20 22 90.91%
src/model.jl 31 34 91.18%
ext/DynamicPPLJETExt.jl 0 4 0.0%
src/debug_utils.jl 24 28 85.71%
src/extract_priors.jl 18 22 81.82%
src/logdensityfunction.jl 12 21 57.14%
Files with Coverage Reduction New Missed Lines %
src/context_implementations.jl 1 93.44%
src/debug_utils.jl 1 89.81%
src/test_utils/ad.jl 1 0.0%
src/test_utils/contexts.jl 1 88.89%
src/contexts.jl 2 74.45%
src/logdensityfunction.jl 2 58.82%
src/utils.jl 2 74.06%
src/distribution_wrappers.jl 6 28.57%
src/simple_varinfo.jl 7 71.76%
src/varinfo.jl 8 85.36%
Totals Coverage Status
Change from base Build 16174747412: -0.2%
Covered Lines: 3325
Relevant Lines: 4010

💛 - Coveralls

mhauru and others added 5 commits May 2, 2025 14:12
* Release 0.36

* AbstractPPL 0.11 + change prefixing behaviour (#830)

* AbstractPPL 0.11; change prefixing behaviour

* Use DynamicPPL.prefix rather than overloading

* Remove VarInfo(VarInfo, params) (#870)

* Unify `{untyped,typed}_{vector_,}varinfo` constructor functions (#879)

* Unify {Untyped,Typed}{Vector,}VarInfo constructors

* Update invocations

* NTVarInfo

* Fix tests

* More fixes

* Fixes

* Fixes

* Fixes

* Use lowercase functions, don't deprecate VarInfo

* Rewrite VarInfo docstring

* Fix methods

* Fix methods (really)

* Draft of accumulators

* Fix some variable names

* Fix pointwise_logdensities, gut tilde_observe, remove resetlogp!!

* Map rather than broadcast

Co-authored-by: Tor Erlend Fjelde <tor.erlend95@gmail.com>

* Start documenting accumulators

* Use Val{symbols} instead of AccTypes to index

* More documentation for accumulators

* Link varinfo by default in AD testing utilities; make test suite run on linked varinfos (#890)

* Link VarInfo by default

* Tweak interface

* Fix tests

* Fix interface so that callers can inspect results

* Document

* Fix tests

* Fix changelog

* Test linked varinfos

Closes #891

* Fix docstring + use AbstractFloat

* Fix resetlogp!! and type stability for accumulators

* Fix type rigidity of LogProbs and NumProduce

* Fix uses of getlogp and other assorted issues

* setaccs!! nicer interface and logdensity function fixes

* Revert back to calling the macro @addlogprob!

* Remove a dead test

* Clarify a comment

* Implement split/combine for PointwiseLogdensityAccumulator

* Switch ThreadSafeVarInfo.accs_by_thread to be a tuple

* Fix `condition` and `fix` in submodels (#892)

* Fix conditioning in submodels

* Simplify contextual_isassumption

* Add documentation

* Fix some tests

* Add tests; fix a bunch of nested submodel issues

* Fix fix as well

* Fix doctests

* Add unit tests for new functions

* Add changelog entry

* Update changelog

Co-authored-by: Hong Ge <3279477+yebai@users.noreply.github.com>

* Finish docs

* Add a test for conditioning submodel via arguments

* Clean new tests up a bit

* Fix for VarNames with non-identity lenses

* Apply suggestions from code review

Co-authored-by: Markus Hauru <markus@mhauru.org>

* Apply suggestions from code review

* Make PrefixContext contain a varname rather than symbol (#896)

---------

Co-authored-by: Hong Ge <3279477+yebai@users.noreply.github.com>
Co-authored-by: Markus Hauru <markus@mhauru.org>

* Revert ThreadSafeVarInfo back to Vectors and fix some AD type casting in (Simple)VarInfo

* Improve accumulator docs

* Add test/accumulators.jl

* Docs fixes

* Various small fixes

* Make DynamicTransformation not use accumulators other than LogPrior

* Fix variable order and name of map_accumulator!!

* Typo fixing

* Small improvement to ThreadSafeVarInfo

* Fix demo_dot_assume_observe_submodel prefixing

* Typo fixing

* Miscellaneous small fixes

* HISTORY entry and more miscellanea

* Add more tests for accumulators

* Improve accumulators docstrings

* Fix a typo

* Expand HISTORY entry

* Add accumulators to API docs

* Remove unexported functions from API docs

* Add NamedTuple methods for get/set/acclogp

* Fix setlogp!! with single scalar to error

* Export AbstractAccumulator, fix a docs typo

* Apply suggestions from code review

Co-authored-by: Penelope Yong <penelopeysm@gmail.com>

* Rename LogPrior -> LogPriorAccumulator, and Likelihood and NumProduce

* Type bound log prob accumulators with T<:Real

* Add @addlogprior! and @addloglikelihood!

* Apply suggestions from code review

Co-authored-by: Penelope Yong <penelopeysm@gmail.com>

* Move default accumulators to default_accumulators.jl

* Fix some tests

* Introduce default_accumulators()

* Go back to only having @addlogprob!

* Fix tilde_observe!! prefixing

* Fix default_accumulators internal type

* Make unflatten more type stable, and add a test for it

* Always print all benchmark results

* Move NumProduce VI functions to abstract_varinfo.jl

---------

Co-authored-by: Penelope Yong <penelopeysm@gmail.com>
Co-authored-by: Tor Erlend Fjelde <tor.erlend95@gmail.com>
Co-authored-by: Hong Ge <3279477+yebai@users.noreply.github.com>
* Implement values_as_in_model using an accumulator

* Make make_varname_expression a function

* Refuse to combine ValuesAsInModelAccumulators with different include_colon_eqs

* Fix nested context test
Copy link
Contributor

github-actions bot commented Jun 2, 2025

Benchmark Report for Commit 2074657

Computer Information

Julia Version 1.11.5
Commit 760b2e5b739 (2025-04-14 06:53 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 4 × AMD EPYC 7763 64-Core Processor
  WORD_SIZE: 64
  LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Benchmark Results

|                 Model | Dimension |  AD Backend |      VarInfo Type | Linked | Eval Time / Ref Time | AD Time / Eval Time |
|-----------------------|-----------|-------------|-------------------|--------|----------------------|---------------------|
| Simple assume observe |         1 | forwarddiff |             typed |  false |                  8.5 |                 1.5 |
|           Smorgasbord |       201 | forwarddiff |             typed |  false |                653.0 |                41.2 |
|           Smorgasbord |       201 | forwarddiff | simple_namedtuple |   true |                418.0 |                50.0 |
|           Smorgasbord |       201 | forwarddiff |           untyped |   true |               1010.4 |                32.9 |
|           Smorgasbord |       201 | forwarddiff |       simple_dict |   true |               6312.4 |                24.8 |
|           Smorgasbord |       201 | reversediff |             typed |   true |               1480.6 |                27.0 |
|           Smorgasbord |       201 |    mooncake |             typed |   true |               1007.7 |                 4.2 |
|    Loop univariate 1k |      1000 |    mooncake |             typed |   true |               5758.1 |                 3.9 |
|       Multivariate 1k |      1000 |    mooncake |             typed |   true |                972.1 |                 9.0 |
|   Loop univariate 10k |     10000 |    mooncake |             typed |   true |              65063.6 |                 3.6 |
|      Multivariate 10k |     10000 |    mooncake |             typed |   true |               8457.1 |                10.0 |
|               Dynamic |        10 |    mooncake |             typed |   true |                137.2 |                12.3 |
|              Submodel |         1 |    mooncake |             typed |   true |                 12.6 |                 6.4 |
|                   LDA |        12 | reversediff |             typed |   true |               1180.7 |                 1.7 |

Copy link
Contributor

github-actions bot commented Jun 2, 2025

DynamicPPL.jl documentation for PR #901 is available at:
https://TuringLang.github.io/DynamicPPL.jl/previews/PR901/

@mhauru mhauru mentioned this pull request Jun 3, 2025
…952)

* Change `evaluate!!` API, add `sample!!`

* Fix literally everything else that I broke

* Fix some docstrings

* fix ForwardDiffExt (look, multiple dispatch bad...)

* Changelog

* fix a test

* Fix docstrings

* use `sample!!`

* Fix a couple more cases

* Globally rename `sample!!` -> `evaluate_and_sample!!`, add changelog warning
* Move submodel code to submodel.jl

* Remove `@submodel`
* Clean up submodel code, remove 3-arg `_evaluate!!`

* Remove 3-argument `evaluate!!` as well

* Update changelog

* Improve submodel error message

* Fix doctest

* Add error hint for three-argument evaluate!!
* Rework API for AD testing

* Fix test

* Add `rng` keyword argument

* Use atol and rtol

* remove unbound type parameter (?)

* Don't need to do elementwise check

* Update changelog

* Fix typo
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants