HTML Report Updates #572
HTML Report Updates #572
Conversation
This adds thin borders around squares in the color box plots used for per-sequence detail plots in reports. Also adds a small gap between squares, tweaks to the spacing of plaquettes, and grid lines delineating between plaquettes for increased visbility.
Updates the version number in readme and adds information about one possible way to cite pygsti in publications.
This commit adds a few new options for finer grained control of the per-sequence detail visualizations. 1. This adds intra-plaquette grids for easier reading of FPRed experiment designs. These can be toggled on and off with a button. 2. The option to toggle on and off hover labels on boxes. 3. A new visualization option that enables click-to-display of the hover label information in an area off to the side of the figures plot area. This can also be toggled on and off. Additionally there is some clean up for long since out of data code. This also adds a copy of the open sans true type file to the repo as this seemed to be the most portable solution to the problem of having a font handy and readily findable by pillow for estimating the rendered size of the hover labels (needed to ensure we have enough margin included so that the text doesn't get cut off).
Default behavior is now to have the clickable hover labels on by default.
Adds the generator infidelity to reports, and adds a helper function to optools for calculating this quantity.
Modify the error generator table text to include the generator infidelity contributions and to call out that the other norm is the frobenius norm.
This adds additional annotations to the C and A project plot to more clearly label which triangle is which, and to make the figure more clear generally.
Initial attempt at making the information in the hover labels and click labels for per-sequence-detail-like plots easier to read by making by using f-string alignment jutsu.
This commit overhauls the presentation of the SPAM estimate table and the choi matrix table. This includes the following changes: 1. SPAM and choi matrices are now split into their real and imaginary parts in separate subplots (when complex valued), rather than having their real and imaginary off-diagonals arranged on the upper and lower triangles of a matrix. 2. The HS vector is not included by default, and there is now plotted as a box plot whenever the states and effects are. 3. A bunch of changes to the reporting code to enable support for these changes, including new plot classes meant to wrap together multiple matrix plots, as well as changes to the existing matrix plots to properly support non-square data. 4. Bug fix for the error bars on the HS vector for POVM effects.
This plumbs in errorbars into the plotting routines for SPAM raw estimates and enables viewing them via hovering over boxes.
This commit adds in additional plumbing for the MDC stores generated during the course of GST estimation so that these can be accessed by the report generation, side-stepping the need to rebuild them from scratch when creating ColorBoxPlots. This only provides a small speedup for relatively small max Ls, but a fairly significant one for large max Ls. e.g. report generation for one-qubit and a maximum depth of 2048 is sped up by ~40% with these changes. Even longer max Ls should be sped up by an even larger fraction.
Adds plumbing for using pre-computed MDC store objects in more places in the reports, and fixes the artifacting of these during the GST protocol. This results in a notable performance for report generation.
Minor tweaks to improve report generation performance by switching to using dictionaries for annotation creation, and inlining some numpy functions.
Rework the SPAM and Gate table error bar handling, and re-implement the ability to split off real and imaginary parts for gates. Additionally, add the target HS vector as a column to the SPAM table.
A series of changes related to the rendering of the new per-sequence detail plots when using the new and improved implementation. This was tracked back to behavior related to trying to have use the aspect ratio of the figure used to rescale its size based on the dimensions of the container. This resulted in unexpected scaling behavior where really wide plots got scaled to be too small. The workaround added, since I didn't see an obvious way to address the logic, was to add the plumbing for skipping this aspect ratio locking behavior for these particular figures.
Fixes a bug in the germs details tables reported in #569. The issue was that the wrong model was being pulled from the switchboard as the target model. Also includes and unrelated fix for the generator infidelity in reports which was accidentally pulling in the average gate infidelity.
Reenable support for the predicted RB number table in the gauge invariant error metric section of the reports. This had previously been disabled, but I believe the version of the relevant function for computing this in the pyGSTi rbtheory tools module does work, so this reenables that computation using the newest code path.
Add a new histogram of the per-circuit TVDs to the goodness of fit overview page.
Fix an issue related to not updating all of the parts of the code touched by the change to the return values from iterative GST now including MDC stores for reuse.
| @@ -197,3 +197,29 @@ Questions? | |||
| ---------- | |||
| For help and support with pyGSTi, please contact the authors at | |||
| pygsti@sandia.gov. | |||
|
|
|||
There was a problem hiding this comment.
I forgot I added this, I think I did so in response to a request from @dhothem. We should probably also include the citation for Zenodo.
There was a problem hiding this comment.
For future reference. https://fonts.google.com/noto/specimen/Noto+Sans/license
There was a problem hiding this comment.
For future reference. https://fonts.google.com/specimen/Open+Sans/license
| # (and triggers resize-script creation) | ||
| if orig_width: del fig['layout']['width'] | ||
| if orig_height: del fig['layout']['height'] | ||
|
|
There was a problem hiding this comment.
This change is related to the fix I found for the figure sizing issues I discussed live in the new per-sequence detail plots.
| @@ -2461,6 +2460,9 @@ def __init__(self, ws, fn, *args): | |||
| The arguments to `fn`. | |||
| """ | |||
| super(WorkspacePlot, self).__init__(ws) | |||
| #add a plot specific option for aspect ratios | |||
| self.options['lock_aspect_ratio'] = True | |||
There was a problem hiding this comment.
Adding a handle for this option was the second half of the fix for the sizing issue I reported for per-sequence detail plots.
Set minimum version to 3.8.0
coreyostrove
requested review from
pcwysoc,
a team and
rileyjmurray
as code owners
| #TEMPORARILY disabled b/c RB analysis is broken | ||
| #from ..extras.rb import theory as _rbtheory | ||
| return -1.0 # _rbtheory.predicted_rb_number(model_a, model_b) | ||
| return _tools.rbtheory.predicted_rb_number(model_a, model_b) |
There was a problem hiding this comment.
Ran this change past @tjproct and also asked him to take a look at the rest of the code for the predicted RB number table's construction and no immediate concerns were raised.
|
Hi @coreyostrove, Excited to see these features getting close to the finish line! Here's a provisional list of feedback:
Overall, it looks great and we shouldn't let the perfect be the enemy of the good with regards to implementation. |
|
Thanks for the feedback, @pcwysoc!
That should be simple enough to change.
Is this an HTML report issue, or is it one that goes deeper?
You're correct that a number of the instrument-related updates are on the aforementioned feature-QI-HTML-integration branch and not represented here. I haven't forgotten about those, but there are some instrument related plots that are currently broken on that branch (some issues related to error bars) which will take some effort to get fixed. I don't have a timeline on that, so decided to try and get the changes that were ready integrated in the meantime.
Could you share a screenshot?
This is also on my to-do list for whenever the feature-QI-HTML-integration branch gets merged in. This was listed in one of the open issue threads so I'll make sure that line item doesn't get prematurely deleted so we can track this. |
|
@coreyostrove here's some answers to your questions: The blocking problem on the feature-QI-HTML branch is one and the same as the Raw Estimates error bar issue, which is as follows. Error bars on the quantum instrument process matrices are being calculated by falling back on using 'deriv_wrt_params,' since we don't know how to reshape the interval matrix. TPInstruments are parameterized as Q0 (fully parameterized) and Q0+Q1 (full TP parameterization). The error bars on Q1 are then effectively calculated summing the error bars on the Q0 and Q0+Q1 parameters. Consequently, the error bars on the second process matrix will spuriously be roughly twice those of the first process matrix as the positive covariance between the Q1 and Q0+Q1 parameters is ignored. One way to avoid this issue is to calculate the error bars directly from the confidence region interval as one might a reportable quantity. It may also be possible to work out how the interval matrix should reshaped, which would be the optimal fix. Here's a screenshot of the error generator metrics issue: 
|
This PR introduces a number of changes to the display, functionality and performance of HTML report generation.
standard_report_high_depth.zip
instruments_and_CIs.zip
Visualization Changes (Existing Figures)
I will describe the changes briefly below, but it is likely most informative to see these in-situ so I have attached two demo 1Q GST reports, one with a really high maximum depth and another containing both instruments and error bars.
New Stuff
Here are some new additions to the reports.
Performance Changes
100% honesty, report generation will often be at least a bit slower on the whole due to the changes on this branch. A lot of that has to do with the fact that we're now drawing a lot more geometry for the per-sequence details. There are some changes to the report generation that speed certain subroutines up and ameliorate this increased time cost quite a bit, though. The biggest change is completing some plumbing for reusing MDC store objects that are generated in the course of GST's iterative optimization to accelerate the construction of certain plots which rely on them (and previously were rebuilding new ones from scratch). If my commit messages from a year ago are to be trusted then for smaller maximum circuit depths the speedup is rather modest, but for a maximum depth of 2048 with one-qubit GST this change sped up report generation by ~40%.
Bugfixes