Bring back sort:indexed for code searches

[svennergr]

Select Topic Area

Product Feedback

Body

On March 10th you informed public that you'll deprecate the sort parameter for code searches: https://github.blog/changelog/2023-03-10-changes-to-the-code-search-api/

I think this was done quite quickly - only 30 days after the deprecation notice.

Are there any plans to bring this feature back/to the new search API?

I think this is a very commonly used feature, which helps users to see most recent code changes containing some search term in a repo.

[clayne]

Yes, please bring this back. It makes it a total hassle to search for only recently updated repositories meaning both the client and server side are having to do excessive amounts of work and data transfer when the client code doesn't even want that.

Example: 10000 results, one only cares about the last 100 recently updated. With prior model this was easy. With current it forces the client to fetch the entire result set and then post-filter. Total regression.

[carcabot]

Wtf, you have just broke the entire GitHub usability without a concrete search method like sorting code by date. Competition is waiting for such mistakes.

[nashbuddhav]

Must have feature

[mitzk215]

indeed a very common and useful feature that must be present !

[DeathKnight813]

Good Advice

[tglam03]

Very Good

[Damian89]

This is one of the worst decisions github made... I am using this feature extensively. If they drop it, code search is just garbage search.

[deenrookie]

This is one of the worst decisions github made... I am using this feature extensively. If they drop it, code search is just garbage search.

I can't agree more

[Milan-960]

Indeed, I agree! It has also removed Sort by: From the code search!

[Aidils60]

super angreeee

[Marius-Ar]

Yes, made CS almost unusable 😢

[DarkReaperBoy]

yeah i need to find shadowsocks config

[kero3009destiny]

Maybe openly admitting to misuse the feature won't help bring it back.

[Damian89]

Not that they removed a useful feature - the new version outputs no data... besides that... I dont want to see 10 year old code as example for anything. this sucks massively... but I dont think we will see this feature again, Microsoft doesnt care.

[alexanderbkl]

I'm using it, it's incredibly awful that they removed that feature.

[trompx]

Besides not being able to sort by recency, now if you want to see when the file has been updated, you have to hover the filename.
For some searches, I want to see how developers are solving problems nowadays, not 10 years ago.
No more number of matching files per language -> very bad for discoverability.
Not possible to filter by multiple languages anymore.
Such a downgrade. That's so annoying and make it totally unusable.

[colinwm]

Everyone, thanks for the feedback. We've definitely heard that sorting by date is a feature many people miss from the old code search.

Unfortunately due to the way our new code search index works (which you can read about in some detail in this blog post), implementing sorting by recency is complex.

We are thinking about way to make this work in the new code search system, but don't have a date for when it will be possible. I'll update again here once we have a more concrete plan.

[dklenowski]

don't get it? your not "sorting by recency" on the whole corpus, just the results.

[uBadRequest]

Hopefully you guys are saving lots of money with the drop in people making searches.😂 I wish I could revert back to old code search.

[Abd3lh4di]

Please turn on the old search feature again, Github became useless to me as I only get result from very old code. please do something!

[anthonyalayo]

Can we get an update?

[johndpope]

@colinwm - can you maybe hire some people that could help ?
maybe ask chatgpt? maybe open source the search feature?
in sql there's a field "sort by". jeah, that should do it. just use this.

[sixgodfan]

Please allow us to revert to the old search in the meantime +1

[serdkc]

please enable old search .

[Baiyuetribe]

Please turn on the old search feature again

[Jamescordon111]

Please make a way for us to use old functionality of search. As my works is immensely effected by that. It's an humble request 🙏

[dev-oneclick]

Please give us the option to use the old version, this is disturbing a lot, the user can have the option of which of the two models to use.

[unprofitable]

+1, you cant do anything without searching...

[franz101]

At least make it filterable by date

[enriqueg]

Please bring the old search back. Why force everybody to use something that doesn't work and it is a UI disaster :(

[matthewpersico]

It is almost as if people who do NOT program and making these decisions. 🤦‍♂️ https://images.app.goo.gl/ZBprA5JiMh8Pofd3A

[xiekunwhy]

The code search results now are much wores than that in about 2 years ago.

[le2Ks]

Bro, even Microsoft Windows 95 had a better search engine almost 30 years ago.

[k1tzu]

I hate that I have to dig outdated code in search results. This is a real pain Microsoft inflicted on developers.😔

[cobavcova]

Yes, that's right, did Microsoft change people or what?

[le2Ks]

That's because every company nowadays, including Microsoft, go with the DEI (Diversity, equity, and inclusion). They don't hire people based on their performance.

[yashrs]

Not that my comment is going to change anything, but please for the love of God, implement sorting by recency.

[johndpope]

I get some repos are created on github and maybe automatically indexing them causes problems for authors where they thought that the repo would be hidden because it wasn't linked to or somehow they think repo is a secret or they don't enable privacy. then results bubble that can be sensitive.
if this is the reason - solve that problem.
if it's because you want to grant access to partners so they can scan for vulnerabilities - then solve that problem. we just want sorting - that doesn't put 5 year old / out of date code in front of new code / apis.
there must be a way to do both. eg. restrict sort for repos so that they don't index in the first 30 days. make it painfully obvious that new repos are going to bubble up in search unless they make it private.
am I missing something? is there really a problem where you can't actually sort by date? I don't believe this. I can get all the results - get the dates - and order the results.

UPDATE
restrict the search results that can be sorted by repos that have been scanned and git issues raised.
if they're disclosing api keys or other nonsense - then fix that problem.

[LiteBrite82]

@benjsilverstein also submitted a very helpful use case last July. My predecessor had some useful insight to the root cause of the indexing issue. This same thread, July 5, 2023

[cobavcova]

I found a bug, when I search for something, the results don't show anything, please fix it

[LiteBrite82]

I cant tell if this is a sick burn, or a real bug.

I couldn't replicate your issue, can you provide more specifics (search parameters)?

If it is indeed a sick burn; well played, I appreciate you.

[cobavcova]

this is my search

[cobavcova]

[Mistik]

Any updates when this will be fixed?

[demo20230423]

well, a year passed, still keep mess/garbage
give me back old code search!

[johndpope]

My reading is - until we give @LiteBrite82 a valid use case - he can't help us?

Do you need a bigger use case than say - searching for code that's new?
If I'm searching for a paper / code - google scholar bundles results by year. see below screenshot.
Please give us this ability to at least do this - it's painful seeing results from 6 years ago for code that is prehistoric (in the top results) and no longer relevant. We're dealing with libraries that go out of date - get superceded. eg. i'm not interested in machine learning code that's built in theano - caffe - I need results that are current.

[cobavcova]

@johndpope It's not Github. Github was for us - developers. But now Microsoft trying to make the life of developers harder for some profit because it's a for-profit company. Remind me of Disney in some way.

[rickgm]

Is it still the case that "GitHub Code Search will only index repositories that have had recent activity within the last year"?

[look]

No, that changelog entry applied to the old code search system. The new system doesn't include every repository, but if you search a repository that isn't included, it will be added to the index and kept up to date.

[rickgm]

Thanks. So you are saying that if you search all the repos under an org "org:blah meow" that the search will apply to all repos in the org and not just repos that have activity in the last year?

[look]

Yes, it will include all the repositories in blah that are indexed.

[immensespace]

It's been over a year since this has been posted, are there any updates as to why we can't use sort:indexed?

Thanks for this context @benjsilverstein. It sounds like you have a lot of duplicated code and therefore finding the most recent examples is very useful to you. It's really helpful to hear examples like this to prioritize work on sorting.

We have definitely heard loud and clear that people want to be able to sort by recency. As I've mentioned in previous responses, the way it was implemented previously does not work for our new code search, because we rebuild the index frequently and the initial build would give everything the same date. We would like to implement this using the time the file was committed, but that is not easy to access in Git due to how data is stored in a repository.

@look posted that it's been heard loud and clear, and that was in July of 2023. If anything, I assume it's been heard louder and clearer in the year since.

For my use cases, I often look at APIs that change rapidly. It's therefore incredibly helpful to be able to sort by date, as often different repos are using different APIs. It's useless to look at a repo that's 2 years or older, and often the new code search is giving me results from 5 years ago.

[look]

Thanks for describing your use case, that's helpful for us to understand how people would like to use date-based sorting. This hasn't been implemented because it's difficult to implement correctly. I posted some details about that here: https://github.com/orgs/community/discussions/121777#discussioncomment-9375680

[le2Ks]

They will never fix it bro...

[zb3]

This hasn't been implemented because it's difficult to implement correctly.

Is there anything that is easy to implement correctly, given GitHub scale? Are you saying that this problem is too difficult for Microsoft? Haha, come on.. Mentioning technical difficulties here is kinda disrespectful..

Now, obviously, I see, this costs. For me, code search is the most useful feature here, so advanced search could even be a paid feature that's part of the core of the product!

[sleep-ops]

I'd be happy to implement it

[alechemy]

Until sort by date is restored, it'd be great to at least show the date in the results (I would hope that this is not also a heavy technical burden).

Right now, I have no way of knowing whether the code is 5 weeks, 5 months, or 5 years old until I click on a result.

[johndpope]

... or somebody build a browser plugin to go off and load each individual results

• come back with the date - and show in some sort of popover.

But seriously - is there a timeline to completely replace search architecture with an LLM across all code ?

[Mistik]

bring it back..

[Sergei-TOL]

bring it back +1!!!

[NoPurposeInLife]

why is this even removed in the first place??? bring it back !