Update stats script

[ddbeck]

Towards #3510, this PR updates ./scripts/stats.ts to return a JSON object with counts of:

• The total features
• The total groups
• The total compat keys in our BCD dependency
• The unmapped compat keys
• The unmapped compat keys that are normal (i.e., not nonstandard or deprecated in BCD)
• The unmapped compat keys that are discourageable (i.e., nonstandard or deprecated in BCD)
• The cumulative shipping days for each (i.e., for each requisite compat key, how many days has a browser shipped that feature in a stable release; see Generate new statistics #3510 (comment) for more details)
• The total caniuse IDs in our caniuse-lite dependency (excluding some dupes of BCD)
• The unmapped caniuseIds

For sample output, see #4030 (comment).

(Yes, some of those numbers are very large! This is expected.)

If you rerun it with --previous some-previous-run.json , it'll include a change key with the difference between the two runs.

Possible follow up work:

• Create a stats.json release artifact for comparing stats over time
• Create a regularly-posted comment thread similar to Feature statistics #788, possibly with graphs

[ddbeck]

This ready for review again.

[Elchi3]

Thanks Daniel, this works as described. You may need to merge latest main again to see if it still functions.

I have to say the meaning of the cumulative days is not obvious to me. I have nothing to put it in relation to.

Inspired by openwebdocs/mdn-bcd-collector#3249, I would have proposed:

Counts:

• featuresCount
• groupsCount
• compatKeysCount
• mappedCompatKeysCount
• unmappedCompatKeysCount
• unmappedNormalCompatKeysCount
• unmappedDiscoragedCompatKeysCount
• caniuseIdsCount
• unmappedCaniuseIdsCount

Ratios:

• mappedCompatKeysRatio
• unmappedCompatKeysRatio
• unmappedNormalCompatKeysRatio
• unmappedDiscoragedCompatKeysRatio
• unmappedCaniuseIdsRatio

[ddbeck]

OK, I've pushed a change to regularize the names (as you suggested @Elchi3) and just go ahead and provide up-and-down measures for everything but the ratios (i.e., you get mappedCompatKeysCount and unmappedCompatKeysCount).

I also commented the fields more, to provide some explanations.

The hardest one is cumulative shipping days. The shortest summary I can think of is: it's a number that gets bigger for every day when we have unmapped keys, but it's exaggerated for keys that are implemented in multiple browsers. Here's my longer description of cumulative shipping days:

The "cumulative shipping days" metrics are the sum of days since each BCD key has shipped in each browser in the core browser set. A smaller number is better. This number increases every day for every key that has shipped but has not been mapped. This metric encourages us to map keys that are more likely to affect real-world developers (i.e., features implemented in multiple browsers). Widely-implemented but not mapped keys increase this metric more (up to +7 per calendar day per key), while other keys increase this metric less or not at all. Keys which have not yet shipped in any stable release count as 0. Like the Baseline calculation, partials and prefixes do not count as shipping.

Here's a new example output:

example.json

{
  "featuresCount": 1168,
  "groupsCount": 103,
  "compatKeysCount": 17734,
  "normalCompatKeysCount": 16042,
  "discourageableCompatKeysCount": 1692,
  "mappedCompatKeysCount": 15060,
  "unmappedCompatKeysCount": 2674,
  "mappedNormalCompatKeysCount": 13301,
  "unmappedNormalCompatKeysCount": 1759,
  "mappedDiscourageableCompatKeysCount": 777,
  "unmappedDiscourageableCompatKeysCount": 915,
  "mappedCompatKeysRatio": 0.8492161948798918,
  "unmappedCompatKeysRatio": 0.15078380512010828,
  "unmappedNormalCompatKeysRatio": 0.10964966961725471,
  "unmappedDiscourageableCompatKeysRatio": 0.5407801418439716,
  "caniuseIdsCount": 583,
  "unmappedCaniuseIdsCount": 236,
  "unmappedCaniuseIdsRatio": 0.40480274442538594,
  "unmappedCompatKeysCumulativeShippingDays": 31138557,
  "unmappedNormalCompatKeysCumulativeShippingDays": 12107641,
  "unmappedDiscourageableCompatKeysCumulativeShippingDays": 19030916,
  "change": {
    "featuresCountChange": 0,
    "groupsCountChange": 0,
    "compatKeysCountChange": 0,
    "normalCompatKeysCountChange": 0,
    "discourageableCompatKeysCountChange": 0,
    "mappedCompatKeysCountChange": 0,
    "unmappedCompatKeysCountChange": 0,
    "mappedNormalCompatKeysCountChange": 0,
    "unmappedNormalCompatKeysCountChange": 0,
    "mappedDiscourageableCompatKeysCountChange": 0,
    "unmappedDiscourageableCompatKeysCountChange": 0,
    "mappedCompatKeysRatioChange": 0,
    "unmappedCompatKeysRatioChange": 0,
    "unmappedNormalCompatKeysRatioChange": 0,
    "unmappedDiscourageableCompatKeysRatioChange": 0,
    "caniuseIdsCountChange": 0,
    "unmappedCaniuseIdsCountChange": 0,
    "unmappedCaniuseIdsRatioChange": 0,
    "unmappedCompatKeysCumulativeShippingDaysChange": 0,
    "unmappedNormalCompatKeysCumulativeShippingDaysChange": 0,
    "unmappedDiscourageableCompatKeysCumulativeShippingDaysChange": 0
  }
}

[ddbeck]

Also, in writing the cumulative shipping days description, I did notice that I need to change it to not use a floating today value, but rather bind it to the BCD release "clock" (like we do for the Baseline low date calculations). I'll pick that up tomorrow. (Fix goes in this file, doing something like this line.)

[Elchi3]

Thanks Daniel! Great work! This looks good to me!

(It is a bit slow on my machine, but unmapped-compat-keys just takes a while to compute, I assume).

Should it be available as "npm run stats"?

[ddbeck]

Thanks Daniel! Great work! This looks good to me!

Thank you!

(It is a bit slow on my machine, but unmapped-compat-keys just takes a while to compute, I assume).

It is slow, partly because this ends up computing each feature's status twice. I'd like to find the time soon to do a light refactor to memoize some of compute-baseline and a cache the complete index.ts output. Something for another PR, unfortunately.

Should it be available as "npm run stats"?

Yes, that's a good idea. I'll add that and merge.