Add begin-less and end-less ranges

[asterite]

Fixes #7170

This PR introduces begin-less and end-less ranges. If most of us think this is a nice and useful feature we could merge it.

An end-less range is just a range with nil as its end. It can be written as (3..) or (3..nil), but of course without an explicit nil is nicer.

A begin-less range is just a range with nil as its beginning. It can be written as ..3 or nil..3.

These ranges have two semantics:

As a range

A range has methods like each, step, etc. An endless range doesn't stop iteration (well, it will, eventually, when it overflows):

(3..).each do |x|
  puts x
  break if some_condition
end

You can't invoke each on a begin-less range (it errors at compile-time, unless the begin is a nilable int, and in that case it raises at runtime). But you can invoke reverse_each:

(..3).reverse_each do |x|
  # ... eventually yields 0, -1, -2
end

Note that a begin-less range isn't equivalent to a range that starts with 0 in these case.

A range also has a === method which can be use in case (and soon in some Enumerable methods ). In this case an endless range means "greater than the beginning of the range":

number = rand(1..10)
case number
when ..3 then "foo"
when (3..5) then "bar"
when (6..) then "bar" # or just else
end

(well, maybe the above isn't a good example because there's still an else nil branch there)

Another example:

numbers = [1, 10, 3, 4, 5, 8]
numbers.select(6..) # => [10, 8]
numbers.select(..6) # => [1, 3, 4, 5]

The above is the same as:

numbers.select(&.>=(6))
numbers.select(&.<=(6))

I think the former is a bit more readable.

There's also Number#clamp, so now there's another way of doing this:

num = some_number
num = 10 if num <= 10

We can do:

num = num.clamp(10..)

Same goes with capping to a max.

In Ruby 2.6 there was also an example like this:

[1, 2, 3].zip(6..) # => [[1, 6], [2, 7], [3, 8]]

which is nice (no need to explicitly specify an upper bound) but currently doesn't work in Crystal (but I might make it work soon).

As an indexer

As an indexer, an endless range means "until the end of the collection". It's equivalent to passing -1, except that you don't have to write that and you don't have to care about .. vs. ...:

ary = [1, 2, 3, 4, 5]
ary[2..] # => [3, 4, 5]
ary[2...] # => [3, 4, 5]

This works for any indexable (Array, Deque) and String.

For a begin-less range it's equivalent to passing 0, but maybe it's a bit more concise:

ary = [1, 2, 3, 4, 5]
ary[..2] # => [1, 2, 3]

The catch-all range

Because the begin and end are now optional, we can also write (..). It doesn't have much use, really. We could forbid it. But right now you can use it:

# replace all elements of an array with a single element
ary = [1, 2, 3, 4, 5]
ary[..] = 3
ary # => [3]

# replace all elements of an array
ary = [1, 2, 3, 4, 5]
ary[..] = [6, 7, 8]
ary # => [6, 7, 8]

# fill an array with a value
ary = [1, 2, 3]
ary.fill(8, ..)
ary # => [8, 8, 8]

# another way to dup an array
ary = [1, 2, 3]
b = ary[..]

And also:

case number
when .. then "duh"
end

Yeah... not a lot of uses, but the semantics are consistent.

Implementation details

Ideally I would change the Range(Int, Int) restrictions to Range(Int?, Int?). However, that gives a compile-error in some cases saying that you can't have Int inside a union type. Instead of trying to fix that I decided to just drop those inner restrictions for two reasons:

• I don't have much time to think about fixing that error
• It doesn't matter much because you will still get an (probably uglier) error if you pass a non-number range, but:
• I don't think there will be many there passing ranges of non-integers to these methods
• It's still not clear what we want to do with these restrictions: is a Range(Int, Int) actually a thing, or should it be more like Range(< Int, < Int) saying "anything that inherits Int"?

Changing the parser was relatively simple.
Making the changes to Range methods was also simple, but it took a bit of time.
Changing the logic of indexers (ary[3..]) was very easy because all of that logic is in Indexable.range_to_index_and_count (we might want to move that method to Range because it's also used in String which is not an Indexable)

Thoughts?

[j8r]

What's the supposed output of (..) and (...)?

[asterite]

@j8r .. and ...

[straight-shoota]

Range doesn't have to be restricted to Int anyway. Theoretically it can be used with any Comparable that implements #succ.

[j8r]

I mean col[..] is supposed to return the whole collection, col[...] all but the last one? I've seen a spec for nil..nil but not for nil...nil

[asterite]

.. and ... are the same. Inclusive and exclusive against infinite is infinite.

[j8r]

@asterite We have to do ..-2 instead of ... to have all but the last element?
Because we have:

a = [0, 1, 2]
puts a[0...-1] #=> [0, 1] 
puts a[0..-2]  #=> [0, 1]

... won't be equal to [0...-1], but equal to [0..-1] and .. then?

[asterite]

Your reasoning is correct. However an endless range means "up to infinite" or without an end. In the case of indexing a collection, because the size is finite, it just means to take everything available. There the exclusive/inclusive distinction doesn't make sense.

Maybe to prevent this small confusion we could make an endless exclusive range a syntax error. Thoughts?

[j8r]

Yes the nillable end of exclusive ranges could be forbidden (like now).

[sudo-nice]

@asterite There is an example above you wrote:

(3..).reverse_each do |x|
  # ... eventually yields 0, -1, -2
end

Did you really mean it should yield 3, 2, 1, 0, -1, -2, ... MINUS INFINITY?

[sudo-nice]

So that .reverse_each on end-less range would mean start when the range starts, but go the opposite direction?

[asterite]

Oops, I had a typo in the example. Now it's fixed.

[yxhuvud]

Regarding the catch-all range, while it isn't amazing for one dimensional arrays, it will probably be a godsend to people implementing matrices with multiple dimensions.

[RX14]

Can we have a spec for the reverse iterator working too?

[asterite]

Unless I'm misunderstanding something, this is the spec for the reverse iterator. Or did you mean something else?

[RX14]

This is the spec for the reverse_each iterator raising, but not actually working (calling .next)

[asterite]

Ooooh... you are right! In fact this wasn't working. I added a spec and also fixed the code.

[asterite]

I originally had two commits, one for end-less range and another for begin-less range. I'll add more commits that fix the existing issues and then we can squash it into a single commit, after all the changes aren't very big.

[asterite]

@RX14 All comments addressed (except one I didn't understand).

Given that this PR also adds specs for Range#step, I'd like to merge this first and then I can send a PR fixing it for negative steps (it's very similar to with positive steps except we have to use pred).

[bew]

Is foo ..2 the same as foo(..2) or foo..2 ?
(this is important for method calls when generated from macros for example)

[asterite]

@bew Excellent question!

I decided it's parsed as foo..2, so as a range, not as foo(..2). I think parsing it the other way would mean a breaking change, plus you can always disambiguate the two cases with parentheses.

But we can discuss it, of course.

[bew]

Yes, foo..2 is the most logical and easiest to parse from the compiler point of view, but for us humans it's not that easy and logical I think..

The following thought is for ranges in general:
I would prefer to make 1 .. 3 an error, and force the 1..3 or 1..(2 + 1) (no spaces around ..) for ranges.
btw, iirc I never saw ranges with spaces

This way you would easily disambiguate the 2: when there is no space it would ALWAYS be a range, when there is spaces it can only be a partial range (begin and/or end -less range)

This can probably wait a bit, I'll open a discussion about this when this PR is merged

[asterite]

Can anyone reproduce the error in linux 64 bits? I'm having a bit of trouble trying to reproduce it...

[j8r]

No problem on my side with the feature/xless-ranges branch. I've ran make before doing this.

/app # ./bin/crystal build  -o .build/std_spec spec/std_spec.cr?
Using compiled compiler at `.build/crystal'
/app # uname -a
Linux f5b436f201b2 4.18.0-11-generic #12-Ubuntu SMP Tue Oct 23 19:22:37 UTC 2018 x86_64 Linux
/app # cat /etc/alpine-release 
3.9.0
/app # git describe --tags --long --always
0.25.0-579-gcbf479d31

[asterite]

@j8r Thank you! But that's not exactly the command that's run. The full command is:

./bin/crystal build -D preview_overflow -D compiler_rt  -o .build/std_spec spec/std_spec.cr

Check the -D flags there.

[j8r]

No problem too @asterite :

./bin/crystal build -D preview_overflow -D compiler_rt  -o .build/std_spec spec/
std_spec.cr
Using compiled compiler at `.build/crystal'

[asterite]

Hmm... I don't know what's going on then...

[asterite]

(@j8r thank you!)

[asterite]

CI passed this time after merging #7397 🎉 😁

[asterite]

Merge? 😺

[straight-shoota]

Thanks @asterite!