Consistently show or hide warning message from Vega-Lite

[joelostblom]

I noticed Vega-Lite warnings are hidden when using the default html renderer but visible when renderer with vl-convert.
I think we should make it consistent so that they are either always hidden or always visible.
I am not sure if the VL warnings are mostly helpful or mostly a nuisance at the Altair level. E.g. the warning in the example below suggest that we shouldn't use a quantitative variable for faceting which on the one hand sounds like sound advice, but also works without any issues and seems reasonable if it is intentional although maybe changing to nominal is better for some reason and more future proof?
Probably a good idea to show the warnings?

import altair as alt
import pandas as pd


source = pd.DataFrame(
    [
        {"a": "a1", "b": "b2", "c": "z", "p": "0.55"},
        {"a": "a1", "b": "b3", "c": "y", "p": "0.58"},
    ]
)

chart = alt.Chart(source, width=60, height=alt.Step(8)).mark_bar().encode(
    alt.Y("c:N").axis(None),
    alt.X("a:N").title("Factor A"),
    alt.Row("p:Q").header(format=".0%"),
)

[jonmmease]

In the vl-convert case, I believe these warnings are written to standard error by Vega-Lite inside the deno runtime. I'm actually not sure off hand how to intercept them, but it's probably doable.

[joelostblom]

As I'm using vegafusion more, I'm noticing that some of these warnings are quite informative. For example, I didn't know that vega-lite automatically switched a line mark with an x2 encoding to a rule mark:

And still, I haven't run into an instance where the warning helped me do something I couldn't do earlier, it's just more information of what is happening under the hood (here, automatically switching the mark).

[joelostblom]

@jonmmease I was playing around with this a bit again now that warnings will be displayed by default in vl-convert 2.0. I think the warnings can be really useful in Altair (particularly those from the VL compilation) and ideally I would want us to display them all by default, but that is tricky due to the heterogeneity in how they are captured, which I try to summarize in this table:

Altair operation	Vega-Lite compile warnings	Vega runtime warnings
Notebook	No	Browser console
Notebook with VegaFusion	Python vl_convert logger	Browser console
chart.to_dict()	No	No
chart.to_dict(format="vega")	Python vl_convert logger	No
Save as HTML	No	Browser console if opened
Save as svg/png/pdf	Python vl_convert logger	Python vl_convert logger

If possible, I would like us to avoid is that the type of warnings displayed change depending on which renderer and data transformer is active. E.g. if someone creates a chart which doesn't raise any warnings, and then activates VegaFusion for the same chart and suddenly gets a warning; they might think that VegaFusion is the cause of the warning rather than that the warning has always been there and VegaFusion just displayed it.

I wonder if one way to make things more consistent would be to always show VL warnings if vl-convert is installed (with an opt out and leave the Vega warnings for the browser)? At least that would remedy the scenario above. This would involve an additional compile pass through vl-convert for all charts. I benchmarked a bit and it doesn't seem that expensive, around 10-20 ms extra per chart from a warm start compared to the around 10 ms baseline for a regular HTML chart (a cold start is notably more, around 400 ms). I put up a PR

I would be curious to hear your thoughts around this, if you think it is worthwhile trying to make the error reporting more consistent with some solution like this or if there is something more suitable upstream?