spec

There's a full blown test suite for Ruby which the various implementations use: https://github.com/ruby/spec

At one point there was an open source project to formally specify Ruby, but I don’t know if it’s still alive: https://github.com/ruby/spec Hmm, it seems to be alive, but based more on behavior than syntax.

This is a similar situation to Ruby and Rubinius (an alternative implementation of Ruby). Because there was no Ruby specification other than the original MRI implementation, the Rubinius project (a new alternative implementation) created their own test suite to codify expected Ruby behavior. However, the MRI developers didn't use it, and the behavior diverged. The original creator gave up on the idea [0] but it was immediately taken over by others and is still maintained [1]. In case of conflict, though, Matz (lead developer on MRI), not the specification, is the source of truth [2]. [0] https://github.com/rubinius/rubinius-website-archive/blob/87... [1] https://github.com/ruby/spec [2] http://ruby.github.io/rubyspec.github.io/bugs_found/

Java could have been a good example, but Sun had a rather strict validation process for calling something Java. Furthermore, there are big difference in philosophy with C: 1. IB and UB are not considered normal parts of specifications, meaning there's way less opportunity for originality in the interpretation of the specifications 2. there tends to be an ur-implementation, and notable divergences from that tends to be interpreted as either a bug in the other implementation(s) or a lack of specification to be resolved between all implementations Rust only has UB in unsafe (AFAIK), which greatly limits implementation flexibility in terms of observable behaviour; and the reference implementation would very much be considered the reference implementation, so I expect e.g. rust-gcc will be sticking close to the reference implementation and beh

Yes that's https://github.com/ruby/spec - now primarily maintained by TruffleRuby people.

Hi eregon, I've submitted a fix to ruby/spec already! \o/ (I'm @lopopolo on GitHub.) One spec I know is missing is how capture groups should not be expanded into globals on `String#=~`. I'll file an issue [0]. [0] https://github.com/ruby/spec/issues/676

Hi, I'm the author of Artichoke. Ruby has a great black box testing suite called ruby/spec [0] that is shared among multiple Ruby implementations. Artichoke has a custom runner [1] to track progress on implementation completeness. If Artichoke passes ruby/spec and is not compatible with MRI, that's a bug in the specs. re: adding Rust to MRI, I'm working on extracting the mruby backend from the core VM infra + core/stdlib impls [3] which would let us use MRI as the backing VM via rutie. When the Artichoke core and stdlib is complete, this would mean that the MRI runtime could be implemented entirely in Rust [2]. [0] https://github.com/ruby/spec [1] https://github.com/artichoke/artichoke/tree/master/spec-runn... [2] https://github.com/artichoke/artichoke/issues/92 [3] https://artichoke.github.io/artichoke/artichoke_core/

it has indeed changed, the project has been reignited as "The Ruby Spec Suite". It still doesn't cover everything, but it's used and contributed to by multiple implementations, and looks healthy https://github.com/ruby/spec/graphs/contributors

After Rubinius discontinued it as a standalone project, it was adopted by someone else and now lives in the MRI org on GitHub https://github.com/ruby/spec.