Dogen v1.0.10, "Lucira"
Lucira fishing village, Namibe province, Angola. (C) 2016 Rede Angola.
Overview
This sprint brought the infrastructural work to a close. Much was achieved, though mainly relevant to the development process. As always, you can get the gory details in the sprint log, but the below has the highlights.
Complete the vcpkg transition
There were still a number of issues to mop-up including proper OSX build support, removing all references to conan (the previous packaging system) and fixing a number of warnings that resulted from the build settings on vcpkg. We have now fully transitioned to vcpkg and we're already experiencing the benefits of the new package management system: adding new packages across all operative systems now takes a couple of hours (the time it takes to rebuild the vcpkg export in three VMs). However, not all packages are available in vcpkg and not all packages that are available build cleanly on all our supported platforms, so we haven't reached nirvana just yet.
Other build improvements
In parallel to the vcpkg transition we also cleaned up most warnings, resulting in very clean builds on CDash. The only warnings we see are real warnings that need to be addressed. We have tried moving to /W4 and even Wall on MSVC but quickly discovered that it isn't feasible at present, so we are using the compiler default settings until the issues we raised are addressed.
Sadly, we've had to ignore all failing tests across all platforms for now (thus taking a further hit on code coverage). This had to be done because at present the tests do not provide enough information for us to understand why they are failing when looking at the Travis/AppVeyor logs. Since reproducing things locally is just too expensive, we need to rewrite these tests to make them easy to troubleshoot from CI logs. This will be done as part of the code generation of model tests.
A final build "improvement" was the removal of facets that were used only to test the code generator, such as hashing, serialisation etc. This has helped immensely in terms of the build time outs but the major downside is we've lost yet another significant source of testing. It seems the only way forward is to create a nightly build that exercises all features of the code generator and runs on our machines - we just do not have enough time on Travis / AppVeyor to compile non-essential code. We still appear to hit occasional timeouts, but these are much less frequent.
Code coverage
We've lacked code coverage for a very long time, and this has been a pressing need because we need to know which parts of the generated code are not being exercised. We finally managed to get it working thanks to the amazing kcov. It is far superior to gcov and previous alternative approaches, requiring very little work to set up. Unfortunately coverage numbers are now very low now due to the commenting out of many unit tests to resolve the build times issues. However, the great news is we can now monitor the coverage as we re-introduce the tests. Sadly, the code coverage story on C# is still weak as we do not seem to be able to generate any information at present (likely due to NUnit shadowing). This will have to be looked at in the future.
We now have support for both Codecov and Coveralls, which appear to give us different results.
C++ 17 support
One of the long time desires has been to migrate from C++ 14 to C++ 17 so that we can use the new features. However, this migration was blocked due to the difficulties of upgrading packages across all platforms. With the completion of the vcpkg story, we finally had all the building blocks in place to move to C++ 17, which was achieved successfully this sprint. This now means we can start to make use of ranges, string_view and all the latest developments. The very first feature we've introduced is nested namespaces, described below.
Project naming clean-up
Now we've settled on the new standard namespaces structure, as defined by the Framework Design Guidelines, we had to update all projects to match. We've also made the build targets match this structure, as well as the folders in the file system, making them all consistent. Since we had to update the CMake files, we started to make them a bit more modern - but we only scratched the surface.
Defining a Dogen API
As part of the work with Framework Design Guidelines, we've created a model to define the product level API and tested it via scenarios. The API is much cleaner and suitable for interoperability (e.g. SWIG) as well as for the code generation of the remotable interfaces.
User visible changes
The main feature added this sprint was the initial support for C++ 17. You can now set your standard to this version:
#DOGEN quilt.cpp.standard=c++-17
At present the only difference is how nested namespaces are handled. Using our annotations class as an example, prior to enabling C++ 17 we had:
namespace masd
namespace dogen
namespace annotations {
<snip>
} } }
Now we generate the following code:
namespace masd::dogen::annotations {
<snip>
}
Next Sprint
We have reached a bit of a fork in Dogen's development: we have got some good ideas on how to address the fundamental architectural problems, but these require very significant surgery into the core of Dogen and its not yet clear if this can be achieved in an incremental manner. On the other hand, there are a number of important stories that need to be implemented in order to get us in a good shape (such as sorting out the testing story). Hard decisions will have to be made in the next sprint.
Binaries
You can download binaries from Bintray for OSX, Linux and Windows (all 64-bit):
For all other architectures and/or operative systems, you will need to build Dogen from source. Source downloads are available below.