Come up with a draft ordering of the exercises in xJava

[jtigger]

No description provided.

[jtigger]

I'm gonna share my ongoing thoughts, here, as I continue through the track.

• (as I've said in another ticket) gigasecond seems trivial in the Java track.
• accumulate is a fairly simple solution that leverages generics; it's nearly identical to strain. At the very least, they could go as a pair (one right after the other). This should be fairly late in the track as we're drawing on both generics and FP.
• trinary is EXACTLY binary except it's base 3. I think it would be fun to actually deliberately put these back-to-back and challenge the programmer to generalize upto base 16.
• ...

[matthewmorgan]

@jtigger did you see the comment I made on this? It seems to be gone...?

[jtigger]

@matthewmorgan I saw it. That was on a separate issue: #143.

[matthewmorgan]

I'm losing it! thanks :)

[jtigger]

...wait... you had it? ;)

[jtigger]

@exercism/java: the track is about to pick-up this work in earnest.

I've found that taking the time to understand why we're doing a work yields higher quality outcomes.

Please take a moment to consider and share your thoughts: why does the Java track of Exercism exist? What purpose does it serve? Whom does it serve?

[jtigger]

@exercism/java I've created a spreadsheet for us to collect our assessments https://docs.google.com/spreadsheets/d/15UbVn4cBUxI7tekKmNNls9dXsT1rDhtG7Z8lp1nmjlM/edit

Please fill-out the sheet according to your assessment:

• share your thoughts in the Notes tab;
• name the topics invoked by the exercise in the Topics tab;
• estimate the difficulty of the exercise in Difficulty tab.

You might find #134 inspiring.

The original request is #125.

Thanks!

[stkent]

Will take a crack at this over the holidays :)

[jtigger]

FWIW, I've started to settle on an "intrinsic complexity" measure. I see two primary sources of complexity:

1. that arising from required interfaces (i.e. methods and data structures)
2. that arising from required behavior (i.e. operations)

I'm capturing this in another tab in the spreadsheet. If you're interested, take a look. I figure behavior complexity has more of an impact than interface complexity on how hard the problem is, so I put a coefficient in front of them. I'll sort by that complexity rating and see what I get!

Very interested in how others are tackling this problem (of identifying order of exercises).

[jtigger]

Please signal here when you've add your input.

[stkent]

I've added difficulty rankings based on manual inspection of the problem descriptions and test suites. They align fairly well with @jtigger's rankings! I probably won't have time to get to the topics list any time soon - busy times ahead.

[jtigger]

Yeah, thankfully we're not worlds apart! :) I've tossed some stats over the current rankings. For those who haven't voted, you might want to prioritize your votes for the exercises that have a standard deviation of 1 or greater (marked in red).

[stkent]

Great idea!

[jtigger]

Yeah, and "topics" seem like a whole 'nother thing... I'm aware of the general idea of how it can be valuable, but I am not clear one or more use-cases that motivate the use of that data. Also, if a particular exercise aims for a specific topic, shouldn't it be attributed to that exercise (i.e. in x-common)? So, what's the additional topics that are language-specific?

many questions. :)

For that reason, I'm personally interested in just the difficulty, first.

[stkent]

For sure; it's also not clear to me how we handle topics that may or may not touched on by an exercise depending on implementation choices...

Also, I only just realized that we will need to manually decide on an exercise order - the order is not absolutely controlled by difficulty. The difficulty ranking should be a great starting point for that sort, however!

[jtigger]

@FridaTveit, I've created a column for you on the spreadsheet should you want to jump in on this.

[FridaTveit]

Thanks! :)

[jtigger]

At this time, the task at hand is to close the gap on the exercises which have a standard deviation > 1 (in appears with red background in spreadsheet).

I can think of some resolution paths:

1. John and Stuart can share their rationale for their scoring, possibly convincing one or the other to change their rating.
2. Emit, Frida, or Matt could add their vote on these exercises. Presumably, this would close the gap.
3. Ignore the difference and go with the average. To @stkent's point, difficulty does not have to be the only variable in ranking.
4. ...

[jtigger]

added matrix, twelve-days, and rectangles to the spreadsheet.

[stkent]

I think getting some sensible order in place is a reasonably high priority; currently, track newcomers are hit with ETL as the second exercise which is probably not a very friendly welcome. I'd support using the existing scores to seed the preliminary ordering, with the understanding that issues/PRs proposing sequencing changes (with reasoning) are both welcomed and encouraged. That iterative philosophy also fits better with the mutation of exercises that will occur as the canonical test suites change.

This assumes that exercism fetch is robust as exercise order changes, i.e. that it fetches "the earliest exercise not yet completed" rather than "the exercise following your most recently-completed exercise in the track sequence"...?

[jtigger]

Well put, @stkent; I wholeheartedly agree.

I'm fairly certain that the CLI works that way. I'll double-check.

[stkent]

Awesome; once you confirm, I'll open up a PR based on the shared spreadsheet!

[jtigger]

Okay... I've confirmed. The next fetched exercise is determined by taking the entire track, subtracting all problems for which the user has submitted and grabbing the top of that list:

https://github.com/exercism/x-api/blob/7919a00f0d266652ad14d22044892b5e13acd51f/api/v1/routes/exercises.rb#L45

This makes re-ordering the track a very safe operation.

---

Current difficulty profile:

Sorted by average score:

Major change, here! Super excited!

[stkent]

That second graph is so purdy! Thanks for checking out the CLI behavior, John - glad that behaves as expected. I'll reply to your question on the difficulty ranking spreadsheet now, and once that's resolved, open up the reordering PR.

[jtigger]

We've applied this first round of ordering... there's a flight of stories around improving the overall experience... weeeeeeeee!

Considering this first move as done.