Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
81 commits
Select commit Hold shift + click to select a range
5d9d868
Delete obsolete initializations
uklotzde Jan 21, 2018
9c13ed4
Count number of Track object instances
uklotzde Jan 21, 2018
cd904af
Fix memory leak: Don't use deleteLater() for temporary Track objects
uklotzde Jan 21, 2018
c2528d6
Minor optimizations in BaseTrackCache
uklotzde Jan 21, 2018
7097985
Delete undefined function
uklotzde Jan 21, 2018
8555edb
Hide private slot
uklotzde Jan 23, 2018
a836d45
Add regression tests for GlobalTrackCache
uklotzde Jan 22, 2018
0ccdc79
lp1744550: Revive cached tracks instead of deleting
uklotzde Jan 23, 2018
56d65b4
Replace Stat counters in GlobalTrackCache
uklotzde Jan 23, 2018
1db98b4
Fix VS2015 build
uklotzde Jan 23, 2018
20dfed2
Fix another use-after-free issue due to race condition
uklotzde Jan 23, 2018
6d854c5
Delete tracks after successfully evicting them from the cache
uklotzde Jan 24, 2018
59f3237
Rename callback function
uklotzde Jan 24, 2018
fbe891c
Destroy the GlobalTrackCache instance only once
uklotzde Jan 24, 2018
b8a4a98
Disable debug logging
uklotzde Jan 24, 2018
cf7e7e9
Merge branch '2.1' into lp1744550_globaltrackcache
uklotzde Jan 24, 2018
4590b70
Minor optimizations
uklotzde Jan 24, 2018
83f988e
Reduce debug log spam
uklotzde Jan 24, 2018
a80e84b
Disconnect all receivers instead of blocking signals before deletion
uklotzde Jan 25, 2018
c7fb846
Encapsulate deletion of Track objects in separate function
uklotzde Jan 25, 2018
61a7673
Save modified tracks when exiting the application
uklotzde Jan 25, 2018
c94df9d
Use deleteAfter() for cached track objects
uklotzde Jan 25, 2018
70c4c2b
Merge branch '2.1' into lp1744550_globaltrackcache
uklotzde Jan 25, 2018
521a382
Verify that post-evict actions are not executed from different threads
uklotzde Jan 25, 2018
458f0e4
Replace std::pair<> with TrackRefPtr
uklotzde Jan 26, 2018
f972ef9
Fix destruction of GlobalTrackCache
uklotzde Jan 26, 2018
e06d531
Fix nasty debug assertion after introduction of TrackRefPtr
uklotzde Jan 26, 2018
75afcd7
Check main thread assumption in evictor callback implementation
uklotzde Jan 26, 2018
3849582
Fix VS2015 build
uklotzde Jan 26, 2018
6c2c700
Don't use GlobalTrackCache from BrowseThread
uklotzde Jan 26, 2018
ae569aa
Simplify TrackRefPtr
uklotzde Jan 27, 2018
a7a41cb
Extend global track cache test harness
uklotzde Jan 29, 2018
f645406
Reliably suppress emitting signals when evicting and deleting a track
uklotzde Feb 3, 2018
ce4c67f
Just block signals, don't disconnect
uklotzde Feb 3, 2018
150c53a
Clarify why race-conditions are expected to occur
uklotzde Feb 3, 2018
eec0682
Delete unused parameter
uklotzde Feb 3, 2018
9ff719e
Inline member function
uklotzde Feb 4, 2018
d78bc08
Simplify lookup of cached tracks
uklotzde Feb 4, 2018
8a22aaf
Merge branch '2.1' into lp1744550_globaltrackcache
uklotzde Feb 5, 2018
d878f94
Only deactivate but do not destroy GlobalTrackCache when exiting
uklotzde Feb 4, 2018
98593a6
Shorten member function name
uklotzde Feb 6, 2018
dec9f65
Remove redundant code and slim down GlobalTrackCacheLocker
uklotzde Feb 6, 2018
9fe2cfb
Avoid move assignment
uklotzde Feb 6, 2018
d4774d7
Replace derived TrackPointer class with a typedef
uklotzde Feb 7, 2018
2da3059
Use workaround to prevent spurious crashes
uklotzde Feb 7, 2018
0bffc94
Reduce complexity of GlobalTrackCache
uklotzde Feb 8, 2018
8691074
Reorder member functions around GlobalTrackCache
uklotzde Feb 10, 2018
c511036
Fix debug assertion: Missing tracks don't have a canonical location
uklotzde Feb 10, 2018
450c092
Use unordered_map/set
uklotzde Feb 10, 2018
06dae38
Drop DbId::toInt()
uklotzde Feb 10, 2018
8dc437a
Export metadata of track if requested by the user
uklotzde Feb 11, 2018
c892866
Show information about deferred/delayed export of metadata
uklotzde Feb 11, 2018
cf00fd8
Don't mark as dirty when modifying transient data
uklotzde Feb 11, 2018
bc9a898
Merge branch 'lp1748758_mark_track_for_export_metadata' into lp174455…
uklotzde Feb 20, 2018
91f49f2
Check for correct thread when accessing the database
uklotzde Feb 20, 2018
d53befb
Update database in main thread when deleting tracks
uklotzde Feb 20, 2018
986473e
Prevent concurrent read/write access of track files
uklotzde Feb 20, 2018
1def331
Merge branch '2.1' into lp1744550_globaltrackcache
uklotzde Feb 20, 2018
53e1981
Merge branch '2.1' into lp1744550_globaltrackcache
uklotzde Feb 20, 2018
4d71f52
Keep the cache locked while saving and deleting tracks
uklotzde Feb 20, 2018
98dc9ea
Don't delete any tracks after the cache has been destroyed
uklotzde Feb 20, 2018
d3b7054
Add comment about (currently) unused function
uklotzde Feb 20, 2018
94d6ad1
Reword comment about expected and handled race conditions
uklotzde Feb 20, 2018
25368a4
remove unused m_unindexedTracks
daschuer Feb 21, 2018
6af8bc5
Remove unneccesary code
daschuer Feb 21, 2018
4a5d770
Improve comments
daschuer Feb 21, 2018
3eff006
Don't throw any exceptions during the deletion of track objects
uklotzde Feb 22, 2018
a5c15de
Relocate cached tracks
uklotzde Feb 22, 2018
95c8fcb
Replace throw() with noexcept
uklotzde Feb 22, 2018
b6e2f63
Delete confusing try/catch block in noexcept function
uklotzde Feb 22, 2018
6c19f05
Do not delete shared track pointers until expired/released
uklotzde Feb 22, 2018
838c668
Improve comments
daschuer Feb 23, 2018
8b5a0eb
Fix saving and deleting tracks at Mixxx shutdown.
daschuer Feb 23, 2018
ac09175
remove redundant TrackPointer temporary
daschuer Feb 11, 2018
b4f1f51
use std::move to avoid reference counting
daschuer Feb 11, 2018
163203c
Avoid locking in inner loop
daschuer Feb 11, 2018
c3337f7
void passing a Track plain pointer
daschuer Feb 11, 2018
c463541
Simplify code and add/reword/fix some comments
uklotzde Feb 24, 2018
4abb74e
Rename member to better reflect its purpose
uklotzde Feb 24, 2018
75c0ece
Add comments about why we are using a plain pointer
uklotzde Feb 24, 2018
2783fde
Reduce visibility of callback functions
uklotzde Feb 24, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion src/encoder/encoderffmpegcore.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,6 @@ EncoderFfmpegCore::EncoderFfmpegCore(EncoderCallback* pCallback, CodecID codec)
{
m_bStreamInitialized = false;
m_pCallback = pCallback;
m_pMetaData = TrackPointer(NULL);

m_pEncodeFormatCtx = NULL;
m_pEncoderAudioStream = NULL;
Expand Down
2 changes: 1 addition & 1 deletion src/library/autodj/autodjfeature.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -278,7 +278,7 @@ void AutoDJFeature::onRightClickChild(const QPoint& globalPos,
Crate crate;
while (nonAutoDjCrates.populateNext(&crate)) {
auto pAction = std::make_unique<QAction>(crate.getName(), &crateMenu);
m_crateMapper.setMapping(pAction.get(), crate.getId().toInt());
m_crateMapper.setMapping(pAction.get(), crate.getId().value());
connect(pAction.get(), SIGNAL(triggered()), &m_crateMapper, SLOT(map()));
crateMenu.addAction(pAction.get());
pAction.release();
Expand Down
51 changes: 33 additions & 18 deletions src/library/basetrackcache.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -141,31 +141,26 @@ void BaseTrackCache::setSearchColumns(const QStringList& columns) {
m_searchColumns = columns;
}

TrackPointer BaseTrackCache::getRecentTrack(TrackId trackId) const {
const TrackPointer& BaseTrackCache::getRecentTrack(TrackId trackId) const {
DEBUG_ASSERT(m_bIsCaching);
// Only refresh the recently used track if the identifiers
// don't match. Otherwise simply return the corresponding
// pointer to avoid accessing and locking the global track
// cache excessively.
if (m_recentTrackId != trackId) {
refreshRecentTrack(std::move(trackId));
if (trackId.isValid()) {
TrackPointer trackPtr =
GlobalTrackCacheLocker().lookupTrackById(trackId);
replaceRecentTrack(
std::move(trackId),
std::move(trackPtr));
} else {
resetRecentTrack();
}
}
return m_recentTrackPtr;
}

void BaseTrackCache::refreshRecentTrack(TrackId trackId) const {
DEBUG_ASSERT(m_bIsCaching);
if (trackId.isValid()) {
auto trackPtr =
GlobalTrackCache::instance().lookupById(trackId).getTrack();
replaceRecentTrack(
std::move(trackId),
std::move(trackPtr));
} else {
resetRecentTrack();
}
}

void BaseTrackCache::replaceRecentTrack(TrackPointer pTrack) const {
DEBUG_ASSERT(m_bIsCaching);
DEBUG_ASSERT(pTrack);
Expand All @@ -180,7 +175,27 @@ void BaseTrackCache::replaceRecentTrack(TrackId trackId, TrackPointer pTrack) co
if (m_recentTrackId.isValid()) {
if (pTrack) {
DEBUG_ASSERT(m_recentTrackId == pTrack->getId());
m_recentTrackPtr = std::move(pTrack);

// NOTE(uklotzde, 2018-02-07, Fedora 27, GCC 7.3.1, optimize=native (-O3), Core i5-6440HQ)
// Using the move assignment operator here will sooner or later
// store a nullptr in m_recentTrackPtr even if both m_recentTrackPtr
// and pTrack contain a valid but different pointer before the
// assignment and the custom deleter is invoked on m_recentTrackPtr
// during the assignment. The issue does not occur when either
// changing the optimization level from 'native' to 'none' or
// when replacing the move assignment with a swap operation (see below).
//
// Fucked up by the optimizer:
// m_recentTrackPtr = std::move(pTrack);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ups. We have many statement like this in the code.
Is the a bug report we can reference? Which places may also be effected?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No. I wasn't able and did not have the time to isolate this issue for a proper bug report.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What means sooner or later? Did this happen with the plain std::shared_ptr or the inherited class or both?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As described, after the assignment the left hand side argument is null, even if the right hand side argument was non-null before the assignment. This is impossible and cannot be caused by a race condition if we assume (and we really should) that std::shared_ptr is thread-safe! It happens with the plain std::shared_ptr, but only once in a while. I was not able to create a simple test case that reproduces the crash.

The optimizer might be confused by the cascaded invocation of const member functions that in the end modify some mutable members. Just an idea, but I cannot proof it.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do I need to mention how many hours I spend to discover this subtle bug? I'm really annoyed about C++ now, totally frustrating.

//
// The workaround:
m_recentTrackPtr.swap(pTrack);
//
// The following debug assertion will be triggered eventually
// without the workaround in place when browsing and editing
// properties of tracks.
DEBUG_ASSERT(m_recentTrackPtr);

if (m_recentTrackPtr->isDirty()) {
m_dirtyTracks.insert(m_recentTrackId);
} else {
Expand Down Expand Up @@ -316,8 +331,8 @@ void BaseTrackCache::updateTrackInIndex(TrackId trackId) {
updateTracksInIndex(trackIds);
}

void BaseTrackCache::updateTracksInIndex(QSet<TrackId> trackIds) {
if (trackIds.size() == 0) {
void BaseTrackCache::updateTracksInIndex(const QSet<TrackId>& trackIds) {
if (trackIds.isEmpty()) {
return;
}

Expand Down
5 changes: 2 additions & 3 deletions src/library/basetrackcache.h
Original file line number Diff line number Diff line change
Expand Up @@ -87,16 +87,15 @@ class BaseTrackCache : public QObject {
void slotDbTrackAdded(TrackPointer pTrack);

private:
TrackPointer getRecentTrack(TrackId trackId) const;
void refreshRecentTrack(TrackId trackId) const;
const TrackPointer& getRecentTrack(TrackId trackId) const;
void replaceRecentTrack(TrackPointer pTrack) const;
void replaceRecentTrack(TrackId trackId, TrackPointer pTrack) const;
void resetRecentTrack() const;

bool updateIndexWithQuery(const QString& query);
bool updateIndexWithTrackpointer(TrackPointer pTrack);
void updateTrackInIndex(TrackId trackId);
void updateTracksInIndex(QSet<TrackId> trackIds);
void updateTracksInIndex(const QSet<TrackId>& trackIds);
void getTrackValueForColumn(TrackPointer pTrack, int column,
QVariant& trackValue) const;

Expand Down
3 changes: 2 additions & 1 deletion src/library/browse/browsetablemodel.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -345,10 +345,11 @@ void BrowseTableModel::trackLoaded(QString group, TrackPointer pTrack) {
}
}
if (pTrack) {
QString trackLocation = pTrack->getLocation();
for (int row = 0; row < rowCount(); ++row) {
QModelIndex i = index(row, COLUMN_PREVIEW);
QString location = getTrackLocation(i);
if (location == pTrack->getLocation()) {
if (location == trackLocation) {
QStandardItem* item = itemFromIndex(i);
item->setText("1");
break;
Expand Down
16 changes: 4 additions & 12 deletions src/library/browse/browsethread.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,6 @@
#include "library/browse/browsetablemodel.h"

#include "sources/soundsourceproxy.h"
#include "track/globaltrackcache.h"
#include "util/trace.h"


Expand Down Expand Up @@ -154,17 +153,10 @@ void BrowseThread::populateModel() {

const QString filepath = fileIt.next();
{
TrackPointer pTrack =
GlobalTrackCache::instance().resolve(
filepath, thisPath.token()).getTrack();
// The GlobalTrackCache is unlocked instantly even if a new track object
// has been created and inserted into the cache. Newly created track
// objects will only contain a reference of the corresponding file,
// but not any metadata, yet. This reduces lock contention on the
// global track cache.

// Update the track object by (re-)importing metadata from the file
SoundSourceProxy(pTrack).updateTrackFromSource();
const TrackPointer pTrack =
SoundSourceProxy::importTemporaryTrack(
filepath,
thisPath.token());

item = new QStandardItem(pTrack->getFileName());
item->setToolTip(item->text());
Expand Down
15 changes: 8 additions & 7 deletions src/library/coverartutils.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -27,18 +27,19 @@ QString CoverArtUtils::supportedCoverArtExtensionsRegex() {

//static
QImage CoverArtUtils::extractEmbeddedCover(
const QFileInfo& fileInfo) {
SecurityTokenPointer pToken(Sandbox::openSecurityToken(
QFileInfo(fileInfo), true));
return extractEmbeddedCover(fileInfo, pToken);
QFileInfo fileInfo) {
SecurityTokenPointer pToken = Sandbox::openSecurityToken(
QFileInfo(fileInfo), true);
return extractEmbeddedCover(
std::move(fileInfo), std::move(pToken));
}

//static
QImage CoverArtUtils::extractEmbeddedCover(
const QFileInfo& fileInfo,
QFileInfo fileInfo,
SecurityTokenPointer pToken) {
auto pTrack = Track::newTemporary(fileInfo, pToken);
return SoundSourceProxy(pTrack).importCoverImage();
return SoundSourceProxy::importTemporaryCoverImage(
std::move(fileInfo), std::move(pToken));
}

//static
Expand Down
4 changes: 2 additions & 2 deletions src/library/coverartutils.h
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,9 @@ class CoverArtUtils {
// Extracts the first cover art image embedded within the file at
// fileInfo. If no security token is provided a new one is created.
static QImage extractEmbeddedCover(
const QFileInfo& fileInfo);
QFileInfo fileInfo);
static QImage extractEmbeddedCover(
const QFileInfo& fileInfo,
QFileInfo fileInfo,
SecurityTokenPointer pToken);

static QImage loadCover(const CoverInfo& info);
Expand Down
6 changes: 3 additions & 3 deletions src/library/crate/cratefeature.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -777,7 +777,7 @@ void CrateFeature::htmlLinkClicked(const QUrl& link) {
}

void CrateFeature::slotTrackSelected(TrackPointer pTrack) {
m_pSelectedTrack = pTrack;
m_pSelectedTrack = std::move(pTrack);

TreeItem* pRootItem = m_childModel.getRootItem();
VERIFY_OR_DEBUG_ASSERT(pRootItem != nullptr) {
Expand All @@ -786,8 +786,8 @@ void CrateFeature::slotTrackSelected(TrackPointer pTrack) {

TrackId selectedTrackId;
std::vector<CrateId> sortedTrackCrates;
if (pTrack) {
selectedTrackId = pTrack->getId();
if (m_pSelectedTrack) {
selectedTrackId = m_pSelectedTrack->getId();
CrateTrackSelectResult trackCratesIter(
m_pTrackCollection->crates().selectTrackCratesSorted(selectedTrackId));
while (trackCratesIter.next()) {
Expand Down
Loading