tachometer

I would rather say: "benchmark with confidence intervals" or maybe "benchmark with comparisons". It's very hard to say much about an absolute number. You need to compare against some alternative or control, and because of CPU load, throttling, GC, and a thousand other variables, you really should be comparing against that control _in the same run_, and importantly round robin across multiple runs to spread out the noise fairly across each implementation. Then once you have a bunch of measurements you have a distribution and shouldn't just take a mean to compare, but should calculate something like the 95% confidence interval. If you see that those confidence intervals overlap, the you might not really know which is faster. If they don't overlap, then you probably do know which is faster. If you have a good benchmark, then running it more

My old team at Google created a tool to help do better browser benchmarking called Tachometer: https://github.com/google/tachometer It tries to deal with the uncertainties of different browsers, JITs, GCs, CPU throttling, varying hardware, etc., several ways: - Runs benchmarks round-robin to hopefully subject each implementation to varying CPU load and thermal properties evenly. - It reports the confidence interval for an implementation, not the mean. Doesn't throw out outlier samples. - For multiple implementations, compares the distributions of samples, de-emphasizing the mean - For comparisons, reports an NxM difference table, showing how each impl compares to the other. - Can auto-run until confidence intervals for different implementations no longer overlap, giving high confidence that there is an actual difference. - Uses WebDriver

Many benchmarking systems face measurement issues that make it difficult to produce solid results. Any given run might not be running on the same hardware, the same OS, built with the same compiler, running with the same runtime, with the same versions of dependencies, with the same system load, at the same temperature, and so on. One robust solution is to instead do pairwise comparisons, many times in a round robin fashion. The results aren't quite as nice to plot, as you don't get a single consistent speed value, but they are much more reliable and true, and you still get useful information, like ">95% chance that this test is at least 20% faster at this commit than at the previous one". A project I contribute to uses this strategy: https://github.com/Polymer/tachometer, but I'd love it if more benchmarks took this approach.

Honestly just by making components and benchmarking them. Webcomponents are too new to really have a solid best practices guide at the moment and there's a lot of varried opinions out there. My main recommendations are to benchmark early and often. Tachometer is a library from the polymer team that I find does this very well: https://github.com/Polymer/tachometer I.e. for testing shadowroot performance, I threw this together: https://github.com/jjcm/shadowTests You can see the results here: https://raw.githubusercontent.com/jjcm/shadowTests/master/re... TL;DR - a custom element with no shadowroot is about twice as fast.

1. Use discriminated union types: https://www.typescriptlang.org/docs/handbook/advanced-types.... - TypeScript these days sticks with the runtime semantics of JavaScript, and discriminated unions are a very natural way to express algebraic data types in JS. 2. Use https://github.com/YousefED/typescript-json-schema to generate a JSON schema, then use a json schema validator to do runtime validation. That's how https://github.com/Polymer/tachometer handles config file parsing and it works very well.