(fix) Search: add proper DateAddedFilter

[ronso0]

This is currently just a text filter, so basically unusable IMO because
a) expcets - delimiter and ISO 8601 (fragments)
b) only search for explicit days/months/years, for example
added:2025-09-22 -> day
added:25-09 -> month
added:12-12 -> 2012-12 or [any year]-12-12

I sometimes want so find tracks before/after a certain date, so here we go..

---

DateAddedFilterNode

New version expects date strings like shown in the library, eg. 25.10.25
(2-digits years are made 2000-based)

It supports single operator queries with =, <, >, <=, >=
added:<03.12.24 = before december 3rd 2024

TODO

• use user locale -> the Date added column uses the system locale, nothing to do
• read user time zone, convert to UTC
• test date locales other than de

Postponed

• use in Analyze and use OrNode for extraFilter in SearchQueryParser::parseQuery() to fix Ocassionally old tracks show up in Analyze view #14873 and make (fix) BaseTrackCache: fix filterAndSort() to avoid unwanted extra tracks with extraFilter #15571 obsolete
  -> addressed in (fix) BaseTrackCache: fix filterAndSort() to avoid unwanted extra tracks with extraFilter #15597
• restore 'month' search, ie. turn added:12.24 into added:01.12.24-31.12.24
  Not sure how this works / is expected to work with short formats like mm-dd-yy. Requires parsing the format strig to find the month position
• range queries like added:01.01.22-23.04.24
  Probably not feasible as the localized short format may be mm-dd-yy.
  We may use -- as delimiter, but that wouldn't be consistent with other range queries (BPM, rating)

[ronso0]

This is the first step towards fixing the bug hacked around by #15571 #15597

[ronso0]

• restore 'month' search, ie. turn added:12.24 into added:01.12.24-31.12.24

This one is a bit tricky, like it's possible but cumbersome to parse the current QLocale::shortFormat (used in the library to display the date).
I'll implement it sooner or later, but IMO this first batch of improvements is mergable without it already.

[ronso0]

So let me try to summarize what I think would be a good first set of fixes:

• use the date format users see in the library (localized, short format)
• support operators < <= > >= (added: equals added:=)

I think that would make it predictable.

A library option to bring iso 2025-11-08 back would be nice.

Why do you think that would be helpful? It's inconsistent with the display.
I think we can make the parser detect both the short dd-MM-yy format as well as ISO with leading yyyy.

[ronso0]

So let me try to summarize what I think would be a good first set of fixes:

• use the date format users see in the library (localized, short format)
• support operators < <= > >= (added: equals added:=)

That would also allow to fix #15586
I'll try to finish it soon so we can squeeze it in 2.5.4

[ronso0]

• use the date format users see in the library (localized, short format)
• support operators < <= > >= (added: equals added:=)

Done.

#15597 makes use of it for the Analyze view and reworks BaseTrackCache::filterAndSort() in order to fix #14873

[daschuer]

A library option to bring iso 2025-11-08 back would be nice.

Why do you think that would be helpful? It's inconsistent with the display.

I mean for all occurrences. It is helpful for users in who likes to use the en-US (untranslated) version, but can't read 22/23/25 as I do. Currently we use the ISO format for History.

[daschuer]

Some issues
added:<26.10.25 reveals also 25.10.25er Tracks.

The search does not follow the library language settings. It is always using the system date and time setting.
I think it is better to uses the same format as in the library.

[daschuer]

Thank you. Not implementing a range, with "-", looks good.

[ronso0]

I'm still unsure what you mean.
In my case my system locale for date (LC_TIME) is de_DE.utf8.
I'm using the untranslated version as well and all dates I see are in de shortfromat dd.MM.yy
This is a shot from a History playlist:

And this is the formatting for the library view:

mixxx/src/library/basetracktablemodel.cpp

Lines 623 to 669 in 9d19b24

	case ColumnCache::COLUMN_LIBRARYTABLE_DATETIMEADDED:
	case ColumnCache::COLUMN_PLAYLISTTRACKSTABLE_DATETIMEADDED: {
	VERIFY_OR_DEBUG_ASSERT(rawValue.canConvert<QDateTime>()) {
	return QVariant();
	}
	// TODO: This is a hot code path, executed very often while library scrolling,
	// and localDateTimeFromUtc is time consuming, probably because,
	// we pass around QDateTime with a wrong time zone set
	QDateTime dt = mixxx::localDateTimeFromUtc(rawValue.toDateTime());
	if (role == Qt::ToolTipRole || role == kDataExportRole) {
	// localized text date: "Wednesday, May 20, 1998 03:40:13 AM CEST"
	return dt;
	}
	if (field == ColumnCache::COLUMN_PLAYLISTTRACKSTABLE_DATETIMEADDED) {
	// Timestamp column in history feature:
	// Use localized date/time format without text: "5/20/98 03:40 AM"
	return mixxx::displayLocalDateTime(dt);
	}
	// For Date Added, use just the date: "5/20/98"
	return dt.date();
	}
	case ColumnCache::COLUMN_LIBRARYTABLE_LAST_PLAYED_AT: {
	QDateTime lastPlayedAt;
	#if QT_VERSION >= QT_VERSION_CHECK(6, 0, 0)
	if (rawValue.metaType().id() == QMetaType::QString) {
	#else
	if (rawValue.type() == QVariant::String) {
	#endif
	// column value
	lastPlayedAt = mixxx::sqlite::readGeneratedTimestamp(rawValue);
	} else {
	// cached in memory (local time)
	lastPlayedAt = rawValue.toDateTime().toUTC();
	}
	if (!lastPlayedAt.isValid()) {
	return QVariant();
	}
	DEBUG_ASSERT(lastPlayedAt.timeSpec() == Qt::UTC);
	// TODO: This is a hot code path, executed very often while library scrolling,
	// and localDateTimeFromUtc is time consuming, probably because,
	// we pass around QDateTime with a wrong time zone set
	QDateTime dt = mixxx::localDateTimeFromUtc(lastPlayedAt);
	if (role == Qt::ToolTipRole || role == kDataExportRole) {
	return dt;
	}
	return dt.date();
	}

What is your LC_TIME locale and what format do you see in the library?

But

I missed that we indeed set the locale for the app (not just the translations) when one is set in the preferences.
Else the system (date) locale is used.
Fixed. For example with Uzbek locale:

Btw we're using the ISO format yyyy-mm-ddTHH:mm:ss.zzz only in the db.

[ronso0]

All roger and ready to roll now IMO, please test

[daschuer]

In my case my system locale for date (LC_TIME) is de_DE.utf8.

This is mine as well. As soon I change the language settings in Mixxx to en_US, the date and all so time format changes also to the (hard to read for me) en_US
format. The search box however still expects the short de_DE format. This is maximum confusing of you are in the UK for instance where day and month are swapped in the very same 10/11/12 notation.

The "issue" (no bug IMHO). because I actually like the DIN recommendation to use the ISO format everywhere, is that the history playlist names are using the iso format.

[ronso0]

is that the history playlist names are using the iso format.

I see. And Iso makes sense for those (it's just the initial name btw) because that's they're sorted by id (which is kind of the same as date_created) so you can easily spot a playlist from let's say 2023-09.

[ronso0]

In my case my system locale for date (LC_TIME) is de_DE.utf8.

This is mine as well. As soon I change the language settings in Mixxx to en_US, the date and all so time format changes also to the (hard to read for me) en_US format. The search box however still expects the short de_DE format. This is maximum confusing of you are in the UK for instance where day and month are swapped in the very same 10/11/12 notation.

To clarify, this are my relevant locale settings:
LANG=de_DE.UTF-8
LANGUAGE=de_DE:en
LC_CTYPE=en_GB.UTF-8
LC_TIME=de_DE.utf8

So with "system" in Mixxx it detects "C" and I get en_US for the translations and the date is de_DE.
If I select any other locale in Mixxx and restart, the translations and date format are adjusted accordingly. The searchbox is consistent with the library.

So, I don't understand why the search would expect de_DE in your case.
Did you try the current state with the fix for that?

[ronso0]

because I actually like the DIN recommendation to use the ISO format everywhere

I'd say this is indeed a personal preference. IMO it makes sense to show dates in local format. For example if you sort tracks by date_added or last_played, users know intuitively at which part of the aa/bb/cc date they need to look to spot a certain range. Like with de_DE I can shrink the date_added column so I see dd.mm... and it's easy to spot tracks from October for example.

If you think a global date format switch makes sense, that's a feature and would go to 2.6 or main anyway.

[daschuer]

I confirm this is fixed now:

If I select any other locale in Mixxx and restart, the translations and date format are adjusted accordingly. The searchbox is consistent with the library.

I have
LANG=de_DE.UTF-8
LANGUAGE=
LC_CTYPE=
LC_TIME=

which is of my knowledge still on default for Ubuntu Jammy.

System

added:>=26.10.25

works.

I'd say this is indeed a personal preference.

Yes of cause. Can you confirm that there is a use case for "international" with en_US translation ans ISO date and time format?
Than I will file a bug for it.

Issue:

added:<26/10/2025

(en_UK)

reveals also 26/10/2025 tracks

added:>=26/10/2025

does nor reveal 26/10/2025

[ronso0]

Issue:

added:<26/10/2025
(en_UK)
reveals also 26/10/2025 tracks

added:>=26/10/2025
does nor reveal 26/10/2025

I improved the debug output, please post what's logged for that query.

edit can ot confirm, works fine here, no matter which locale

[daschuer]

It works here. I have probably whatched the wrong column, sorry.

[ronso0]

Alright, thanks for verifying.
I reverted the trace commit and this is now ready for merge.

I'll open a manual PR soonish.

[daschuer]

There is still an issue with
added:02.10.25 and added:=02.10.25

warning [Main]      .
warning [Main]      DateAdded: "02.10.25"
warning [Main]      .
warning [Main]      .
warning [Main]      DateAdded toSql: ! invalid, return true
warning [Main]    

And added:>=02.10.25 AND added:<03.10.25

warning [Main]      .
warning [Main]      DateAdded: ">=02.10.25"
warning [Main]        opMatch:  ">="
warning [Main]        arg:      "02.10.25"
warning [Main]        parseDate: app locale: QLocale(German, Latin, Germany) "dd.MM.yy"
warning [Main]                system locale: QLocale(German, Latin, Germany) "dd.MM.yy"
warning [Main]        date auto: QDate("1925-10-02")
warning [Main]        date adj.: QDate("2025-10-02")
warning [Main]      date valid, to string: "2025-10-01T22:00:00.000"
warning [Main]      -> opQuery
warning [Main]      .
warning [Main]      .
warning [Main]      DateAdded: "<03.10.25"
warning [Main]        opMatch:  "<"
warning [Main]        arg:      "03.10.25"
warning [Main]        parseDate: app locale: QLocale(German, Latin, Germany) "dd.MM.yy"
warning [Main]                system locale: QLocale(German, Latin, Germany) "dd.MM.yy"
warning [Main]        date auto: QDate("1925-10-03")
warning [Main]        date adj.: QDate("2025-10-03")
warning [Main]      date valid, to string: "2025-10-02T22:00:00.000"
warning [Main]      -> opQuery
warning [Main]      .
warning [Main]      .
warning [Main]      DateAdded toSql: op: "datetime_added >= '2025-10-01T22:00:00.000'"
warning [Main]      .
warning [Main]      .
warning [Main]      DateAdded toSql: op: "datetime_added < '2025-10-02T22:00:00.000'"
warning [Main]      .

Why 22:00:00?

[ronso0]

Why 22:00:00?

Because timestamps are in UTC and you're currently in CET (Berlin, UTC+1) + 1h daylight saving time.
So UTC = local - 2h.
That's the same conversion like for the library if I'm not mistaking.

Will look into the other issue (sadly I already removed the trace commit..)

[ronso0]

Got it!
Now works also with = and pure : (implicit =).
Though I didn't polish, maybe you have ideas to optimize it?

[daschuer]

I think (not tested) one ms slips trough. See suggestions.

[ronso0]

Thanks!
I think all cases are now covered correctly.

[ronso0]

@daschuer this is ready.
Let me know when I can remove the trace commit.

[daschuer]

LGTM, thank you.

[ronso0]

Cool, thanks for testing!

Let me know when I can remove the trace commit.

Done.