Rename example to test case in many places#4631
Conversation
|
Hmm. I feel pretty good about generally tidying up our language and settling on specific names for things, but I think I'd want to have a more complete view before shipping it. Maybe we should write up explicit "user glossary" and "contributor glossary" docs to identify what terms need definitions or explanation? Explaining existing use of terms is already a nice improvement on the status quo; and then after that we can decide whether it's worth changing some of them. (e.g.: I find all of 'example', 'input', and 'test case' natural in slightly different situations 🙈) |
|
My thoughts:
It's tempting to harmonize |
…thesis-4631 # Conflicts: # hypothesis-python/tests/conjecture/test_inquisitor.py # hypothesis/RELEASE.rst # hypothesis/tests/cover/test_custom_reprs.py # hypothesis/tests/cover/test_stateful.py # hypothesis/tests/pytest/test_skipping.py
Liam-DeVoe
left a comment
Liam-DeVoe
left a comment
There was a problem hiding this comment.
@Zac-HD this is ready for review.
I've added a new glossary page, split into "user" and "developer" sections. I added the latter because I wanted a glossary item for things like "choice sequence", but didn't want to imply to users that this was a normal part of hypothesis and our public API.
Because there's a lot of semantically-identical changes spread across files here, I've left comments to highlight the meaningful changes for your review.
I'm quite happy with the "test case" terminology. I know @DRMacIver is as well.
| """ | ||
| Once this many satisfying examples have been considered without finding any | ||
| counter-example, Hypothesis will stop looking. | ||
| Once this many satisfying |test cases| have been considered without finding | ||
| any failing test case, Hypothesis will stop looking. | ||
|
|
||
| .. note:: | ||
|
|
||
| Historically, Hypothesis referred to test cases as "examples", including in | ||
| this setting name. We now refer to them as test cases throughout our | ||
| documentation and codebase. | ||
|
|
||
| This setting is an exception: to avoid ecosystem churn, this setting will | ||
| continue to be called ``max_examples``. Conceptually, it is better thought of | ||
| as "max test cases". |
There was a problem hiding this comment.
note about historical example terminology
|
|
||
| a:has(> .std-term), | ||
| a:has(> .std-term):hover { | ||
| color: inherit; | ||
| text-decoration-color: currentColor; | ||
| text-decoration-style: dotted; | ||
| } |
There was a problem hiding this comment.
combining our css files into one. The new content here is what I've highlighted here: removing blue links and adding underlines to glossary terms, for a more low-key look.
| ... f() | ||
| Trying example: [] | ||
| Falsifying example: [-1198601713, -67, 116, -29578] | ||
| Shrunk example to [-1198601713] | ||
| Shrunk example to [-128] | ||
| Shrunk example to [32] | ||
| Shrunk example to [1] | ||
| Test case: [] | ||
| Failing test case: [-1198601713, -67, 116, -29578] | ||
| Shrunk test case to [-1198601713] | ||
| Shrunk test case to [-128] | ||
| Shrunk test case to [32] | ||
| Shrunk test case to [1] | ||
| [1] | ||
|
|
There was a problem hiding this comment.
this is a nice before/after comparison for your review
2 participants
I'm interested to hear what people think about this one.
A concern I have is
max_examplesbecomes more confusing to users when everything else refers to "test cases". I think this is fine, becausemax_examplesis one of the first keywords a user learns in hypothesis (just after "strategies"), and likely before they encounter "test cases". So the confusion is when they first encounter "test cases", notmax_examples. And we already use test cases in some places, so I mostly view this as pushing the confusion earlier, not new disharmony. (to be clear I don't necessarily think this is good, just not as bad as it could be.)I've also made these two not-strictly-related wording changes (falsifying -> failing, drop "trying"), in trying to give some love to our debug logging:
Falsifying example: test_integers(->Failing test case: test_integers(Trying example: f(->Test case: f(