Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Retry deletion of message if deletion fails #225

Open
wants to merge 15 commits into
base: master
Choose a base branch
from
Open

Retry deletion of message if deletion fails #225

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
f71ac50
add variables to track whether to retry the delete
BepisGithub Jul 11, 2021
c55d220
wrap delete code in a do while loop
BepisGithub Jul 11, 2021
868740a
add wait statement if there is an error before attempting to delete a…
BepisGithub Jul 11, 2021
96162e5
increment deleteErrorCount on each error
BepisGithub Jul 11, 2021
3843f76
rename variable to be more consistent with the project
BepisGithub Jul 11, 2021
1e09cfe
reset delErr value to false on each try of deleting
BepisGithub Jul 11, 2021
4fd8cde
fix incrementing of non existent variable
BepisGithub Jul 11, 2021
71be757
set value of delErr to true if there is an error when trying to delete
BepisGithub Jul 11, 2021
6e429fc
retry deleting a message up to 5 times if it fails
BepisGithub Jul 11, 2021
2b7cc60
fix while loop referencing old variable name
BepisGithub Jul 11, 2021
268c2ea
allows the script to keep on running even if there was an error delet…
BepisGithub Jul 12, 2021
1fe683a
add a 10 second delay between each reattempt of deleting a message
BepisGithub Jul 12, 2021
c10994d
delay.md changes and added an option to adjust retry delay to the GUI
BepisGithub Jul 12, 2021
32a5c62
fix issue where resp could still be undefined
BepisGithub Jul 12, 2021
183f754
wait for the duration of the retry delay if resp is undefined
BepisGithub Jul 12, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
120 changes: 67 additions & 53 deletions deleteDiscordMessages.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -153,6 +153,8 @@
const delayAdjustmentThreshold = 5; // ms
let baseDeleteDelay = 100;
let deleteDelay = baseDeleteDelay;
let baseRetryDelay = 5000;
let retryDelay = 5000;
let baseSearchDelay = 100;
let searchDelay = baseSearchDelay;
let delCount = 0;
Expand All @@ -171,7 +173,7 @@
const redact = str => `<span class="priv">${escapeHTML(str)}</span><span class="mask">REDACTED</span>`;
const queryString = params => params.filter(p => p[1] !== undefined).map(p => p[0] + '=' + encodeURIComponent(p[1])).join('&');
const ask = async msg => new Promise(resolve => setTimeout(() => resolve(popup.confirm(msg)), 10));
const printDelayStats = () => log.verb(`Delete delay: ${deleteDelay}ms, Search delay: ${searchDelay}ms`, `Last Ping: ${lastPing}ms, Average Ping: ${avgPing|0}ms`);
const printDelayStats = () => log.verb(`Delete delay: ${deleteDelay}ms, Search delay: ${searchDelay}ms`, `Retry delay: ${retryDelay}ms,` `Last Ping: ${lastPing}ms, Average Ping: ${avgPing|0}ms`);
const toSnowflake = (date) => /:/.test(date) ? ((new Date(date).getTime() - 1420070400000) * Math.pow(2, 22)) : date;

const log = {
Expand All @@ -197,27 +199,29 @@
};

let resp;
try {
const s = Date.now();
resp = await fetch(API_SEARCH_URL + 'search?' + queryString([
[ 'author_id', authorId || undefined ],
[ 'channel_id', (guildId !== '@me' ? channelId : undefined) || undefined ],
[ 'min_id', minId ? toSnowflake(minId) : undefined ],
[ 'max_id', maxId ? toSnowflake(maxId) : undefined ],
[ 'sort_by', 'timestamp' ],
[ 'sort_order', 'desc' ],
[ 'offset', offset ],
[ 'has', hasLink ? 'link' : undefined ],
[ 'has', hasFile ? 'file' : undefined ],
[ 'content', content || undefined ],
[ 'include_nsfw', includeNsfw ? true : undefined ],
]), { headers });
lastPing = (Date.now() - s);
avgPing = avgPing>0 ? (avgPing*0.9) + (lastPing*0.1) : lastPing;
} catch (err) {
return log.error('Search request threw an error:', err);
}

do{
try {
const s = Date.now();
resp = await fetch(API_SEARCH_URL + 'search?' + queryString([
[ 'author_id', authorId || undefined ],
[ 'channel_id', (guildId !== '@me' ? channelId : undefined) || undefined ],
[ 'min_id', minId ? toSnowflake(minId) : undefined ],
[ 'max_id', maxId ? toSnowflake(maxId) : undefined ],
[ 'sort_by', 'timestamp' ],
[ 'sort_order', 'desc' ],
[ 'offset', offset ],
[ 'has', hasLink ? 'link' : undefined ],
[ 'has', hasFile ? 'file' : undefined ],
[ 'content', content || undefined ],
[ 'include_nsfw', includeNsfw ? true : undefined ],
]), { headers });
lastPing = (Date.now() - s);
avgPing = avgPing>0 ? (avgPing*0.9) + (lastPing*0.1) : lastPing;
} catch (err) {
log.error('Search request threw an error:', err);
await wait(retryDelay);
}
}while(typeof resp === 'undefined');
// not indexed yet
if (resp.status === 202) {
const w = (await resp.json()).retry_after;
Expand Down Expand Up @@ -294,39 +298,49 @@
if (onProgress) onProgress(delCount + 1, grandTotal);

let resp;
try {
const s = Date.now();
const API_DELETE_URL = `https://discord.com/api/v6/channels/${message.channel_id}/messages/${message.id}`;
resp = await fetch(API_DELETE_URL, {
headers,
method: 'DELETE'
});
lastPing = (Date.now() - s);
avgPing = (avgPing*0.9) + (lastPing*0.1);
delCount++;
} catch (err) {
log.error('Delete request throwed an error:', err);
log.verb('Related object:', redact(JSON.stringify(message)));
failCount++;
}

if (!resp.ok) {
// deleting messages too fast
if (resp.status === 429) {
const w = (await resp.json()).retry_after;
throttledCount++;
throttledTotalTime += w;
baseDeleteDelay = deleteDelay;
deleteDelay = w > baseDeleteDelay ? w : baseDeleteDelay;
log.warn(`Being rate limited by the API for ${w}ms! Increasing delete delay to ${deleteDelay}ms, adjusting base delete delay to ${baseDeleteDelay}ms`);
printDelayStats();
log.verb(`Cooling down for ${w*2}ms before retrying...`);
await wait(w*2);
i--; // retry
} else {
log.error(`Error deleting message, API responded with status ${resp.status}!`, await resp.json());
let delErrCount = 0;
let delErr = false;
do{
try {
delErr = false;
const s = Date.now();
const API_DELETE_URL = `https://discord.com/api/v6/channels/${message.channel_id}/messages/${message.id}`;
resp = await fetch(API_DELETE_URL, {
headers,
method: 'DELETE'
});
lastPing = (Date.now() - s);
avgPing = (avgPing*0.9) + (lastPing*0.1);
delCount++;
} catch (err) {
log.error('Delete request throwed an error:', err);
log.verb('Related object:', redact(JSON.stringify(message)));
delErrCount++;
failCount++;
delErr = true;
await wait(retryDelay);
}
}while(delErr && delErrCount < 5); // retry deleting a message up to five times if there's an error

if (typeof resp != "undefined"){
if (!resp.ok) {
// deleting messages too fast
if (resp.status === 429) {
const w = (await resp.json()).retry_after;
throttledCount++;
throttledTotalTime += w;
baseDeleteDelay = deleteDelay;
deleteDelay = w > baseDeleteDelay ? w : baseDeleteDelay;
log.warn(`Being rate limited by the API for ${w}ms! Increasing delete delay to ${deleteDelay}ms, adjusting base delete delay to ${baseDeleteDelay}ms`);
printDelayStats();
log.verb(`Cooling down for ${w*2}ms before retrying...`);
await wait(w*2);
i--; // retry
} else {
log.error(`Error deleting message, API responded with status ${resp.status}!`, await resp.json());
log.verb('Related object:', redact(JSON.stringify(message)));
failCount++;
}
}
}
else if (deleteDelay - baseDeleteDelay > delayAdjustmentThreshold)
Expand Down
125 changes: 72 additions & 53 deletions deleteDiscordMessages.user.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@
* @author Victornpb <https://www.github.com/victornpb>
* @see https://github.com/victornpb/deleteDiscordMessages
*/
async function deleteMessages(authToken, authorId, guildId, channelId, minId, maxId, content, hasLink, hasFile, includeNsfw, includePinned, searchDelay, deleteDelay, extLogger, stopHndl, onProgress) {
async function deleteMessages(authToken, authorId, guildId, channelId, minId, maxId, content, hasLink, hasFile, includeNsfw, includePinned, searchDelay, deleteDelay, retryDelay, extLogger, stopHndl, onProgress) {
const start = new Date();
let delCount = 0;
let failCount = 0;
Expand Down Expand Up @@ -73,27 +73,29 @@ async function deleteMessages(authToken, authorId, guildId, channelId, minId, ma
};

let resp;
try {
const s = Date.now();
resp = await fetch(API_SEARCH_URL + 'search?' + queryString([
['author_id', authorId || undefined],
['channel_id', (guildId !== '@me' ? channelId : undefined) || undefined],
['min_id', minId ? toSnowflake(minId) : undefined],
['max_id', maxId ? toSnowflake(maxId) : undefined],
['sort_by', 'timestamp'],
['sort_order', 'desc'],
['offset', offset],
['has', hasLink ? 'link' : undefined],
['has', hasFile ? 'file' : undefined],
['content', content || undefined],
['include_nsfw', includeNsfw ? true : undefined],
]), { headers });
lastPing = (Date.now() - s);
avgPing = avgPing > 0 ? (avgPing * 0.9) + (lastPing * 0.1) : lastPing;
} catch (err) {
return log.error('Search request threw an error:', err);
}

do{
try {
const s = Date.now();
resp = await fetch(API_SEARCH_URL + 'search?' + queryString([
['author_id', authorId || undefined],
['channel_id', (guildId !== '@me' ? channelId : undefined) || undefined],
['min_id', minId ? toSnowflake(minId) : undefined],
['max_id', maxId ? toSnowflake(maxId) : undefined],
['sort_by', 'timestamp'],
['sort_order', 'desc'],
['offset', offset],
['has', hasLink ? 'link' : undefined],
['has', hasFile ? 'file' : undefined],
['content', content || undefined],
['include_nsfw', includeNsfw ? true : undefined],
]), { headers });
lastPing = (Date.now() - s);
avgPing = avgPing > 0 ? (avgPing * 0.9) + (lastPing * 0.1) : lastPing;
} catch (err) {
log.error('Search request threw an error:', err);
await wait(retryDelay);
}
}while(typeof resp === 'undefined');
// not indexed yet
if (resp.status === 202) {
const w = (await resp.json()).retry_after;
Expand Down Expand Up @@ -164,38 +166,48 @@ async function deleteMessages(authToken, authorId, guildId, channelId, minId, ma
if (onProgress) onProgress(delCount + 1, grandTotal);

let resp;
try {
const s = Date.now();
const API_DELETE_URL = `https://discord.com/api/v6/channels/${message.channel_id}/messages/${message.id}`;
resp = await fetch(API_DELETE_URL, {
headers,
method: 'DELETE'
});
lastPing = (Date.now() - s);
avgPing = (avgPing * 0.9) + (lastPing * 0.1);
delCount++;
} catch (err) {
log.error('Delete request throwed an error:', err);
log.verb('Related object:', redact(JSON.stringify(message)));
failCount++;
}

if (!resp.ok) {
// deleting messages too fast
if (resp.status === 429) {
const w = (await resp.json()).retry_after;
throttledCount++;
throttledTotalTime += w;
deleteDelay = w; // increase delay
log.warn(`Being rate limited by the API for ${w}ms! Adjusted delete delay to ${deleteDelay}ms.`);
printDelayStats();
log.verb(`Cooling down for ${w * 2}ms before retrying...`);
await wait(w * 2);
i--; // retry
} else {
log.error(`Error deleting message, API responded with status ${resp.status}!`, await resp.json());
let delErrCount = 0;
let delErr = false;
do{
try {
delErr = false;
const s = Date.now();
const API_DELETE_URL = `https://discord.com/api/v6/channels/${message.channel_id}/messages/${message.id}`;
resp = await fetch(API_DELETE_URL, {
headers,
method: 'DELETE'
});
lastPing = (Date.now() - s);
avgPing = (avgPing*0.9) + (lastPing*0.1);
delCount++;
} catch (err) {
log.error('Delete request throwed an error:', err);
log.verb('Related object:', redact(JSON.stringify(message)));
delErrCount++;
failCount++;
delErr = true;
await wait(retryDelay);
}
}while(delErr && delErrCount < 5); // retry deleting a message up to five times if there's an error

if (typeof resp != "undefined"){
if (!resp.ok) {
// deleting messages too fast
if (resp.status === 429) {
const w = (await resp.json()).retry_after;
throttledCount++;
throttledTotalTime += w;
deleteDelay = w; // increase delay
log.warn(`Being rate limited by the API for ${w}ms! Adjusted delete delay to ${deleteDelay}ms.`);
printDelayStats();
log.verb(`Cooling down for ${w * 2}ms before retrying...`);
await wait(w * 2);
i--; // retry
} else {
log.error(`Error deleting message, API responded with status ${resp.status}!`, await resp.json());
log.verb('Related object:', redact(JSON.stringify(message)));
failCount++;
}
}
}

Expand Down Expand Up @@ -314,6 +326,12 @@ function initUI() {
target="_blank">?</a><br>
<input id="deleteDelay" type="number" value="1000" step="100">
</span>
</span>
<span>Retry Delay <a
href="https://github.com/victornpb/deleteDiscordMessages/blob/master/help/delay.md" title="Help"
target="_blank">?</a><br>
<input id="retryDelay" type="number" value="5000" step="1000">
</span>
</div>
<hr>
<button id="start" style="background:#43b581;width:80px;">Start</button>
Expand Down Expand Up @@ -387,6 +405,7 @@ function initUI() {
const includePinned = $('input#includePinned').checked;
const searchDelay = parseInt($('input#searchDelay').value.trim());
const deleteDelay = parseInt($('input#deleteDelay').value.trim());
const retryDelay = parseInt($('input#retryDelay').value.trim());
const progress = $('#progress');
const progress2 = btn.querySelector('progress');
const percent = $('.percent');
Expand Down Expand Up @@ -421,7 +440,7 @@ function initUI() {

stop = stopBtn.disabled = !(startBtn.disabled = true);
for (let i = 0; i < channelIds.length; i++) {
await deleteMessages(authToken, authorId, guildId, channelIds[i], minId || minDate, maxId || maxDate, content, hasLink, hasFile, includeNsfw, includePinned, searchDelay, deleteDelay, logger, stopHndl, onProg);
await deleteMessages(authToken, authorId, guildId, channelIds[i], minId || minDate, maxId || maxDate, content, hasLink, hasFile, includeNsfw, includePinned, searchDelay, deleteDelay, retryDelay, logger, stopHndl, onProg);
stop = stopBtn.disabled = !(startBtn.disabled = false);
}
};
Expand Down
5 changes: 4 additions & 1 deletion help/delay.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
# Search and Delete Delay
# Search, Delete and Retry Delay

This setting controls the initial delay for for searching messages. Sometimes the Discord API will quickly rate limit you, in this case setting the delays to something higher might help. This is especially true for the **Delete Delay**.


The retry delay is a value that delays how long the script should wait before trying to delete the same message again if there was an error trying to delete it.