Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add cache-flush step in Twenty upgrade command #7521 #7553

Merged
merged 3 commits into from
Oct 10, 2024

Conversation

yadavshubham01
Copy link
Contributor

@yadavshubham01 yadavshubham01 commented Oct 10, 2024

Fixes: #7521

Flush Specific Redis Keys After Upgrade Command :----

Changes Made :----
Updated the FlushCacheCommand to allow for selective flushing of keys that match the pattern engine:*, rather than flushing the entire cache.
Added a new optional parameter to specify the cache keys pattern.
Ensured that the cache is flushed at the end of the upgrade command.

Code Changes :----
Modified FlushCacheCommand to include a method for flushing keys by pattern.
Added a new function in CacheStorageService to handle the pattern-based flushing.

Copy link

github-actions bot commented Oct 10, 2024

Welcome!

Hello there, congrats on your first PR! We're excited to have you contributing to this project.
By submitting your Pull Request, you acknowledge that you agree with the terms of our Contributor License Agreement.

TODOs/FIXMEs:

  • // TODO: implement dry-run: packages/twenty-server/src/engine/core-modules/cache-storage/commands/flush-cache.command.ts

Generated by 🚫 dangerJS against 178fc27

Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Summary

This PR enhances the cache flushing functionality in the Twenty upgrade command, allowing for selective flushing of Redis cache keys based on a specified pattern.

  • Added flushByPattern method in CacheStorageService to selectively flush Redis keys matching a pattern
  • Updated FlushCacheCommand to accept an optional --pattern flag for targeted cache flushing
  • Implemented pattern-based flushing in FlushCacheCommand, defaulting to full cache flush if no pattern is provided
  • Ensured compatibility with Redis cache, throwing an error for non-Redis caches in flushByPattern
  • Potential performance concern: Use of Redis keys command in flushByPattern may impact large datasets

2 file(s) reviewed, 5 comment(s)
Edit PR Review Bot Settings | Greptile

@@ -20,10 +20,27 @@ export class FlushCacheCommand extends CommandRunner {
) {
super();
}


async run(passedParams: string[], options?: Record<string, any>): Promise<void> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

style: Consider adding type annotations for 'passedParams' and 'options' parameters



async run(passedParams: string[], options?: Record<string, any>): Promise<void> {
const pattern = options?.pattern || '*'; // Default to flushing all keys if no pattern is passed
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

style: Consider using nullish coalescing (??) instead of logical OR (||) for better type safety

Comment on lines +43 to +45
parsePattern(val: string): string {
return val;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

style: parsePattern method can be simplified to a one-liner arrow function

Comment on lines 69 to 82
async flushByPattern(pattern: string): Promise<void> {
if (!this.isRedisCache()) {
throw new Error('flushByPattern is only supported with Redis cache');
}

const redisClient = (this.cache as RedisCache).store.client;


const keys = await redisClient.keys(`${this.namespace}:${pattern}`);
if (keys.length > 0) {
await redisClient.del(keys);
}

}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

style: Consider using SCAN instead of KEYS for better performance in production environments with large datasets

Comment on lines 78 to 80
if (keys.length > 0) {
await redisClient.del(keys);
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

style: Consider using UNLINK instead of DEL for non-blocking key deletion in Redis versions that support it

Copy link
Member

@Weiko Weiko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've updated the PR to use redis SCAN instead of KEYS as this is the recommended way for large dataset due to the fact that KEYS can block the server.
Thanks for your contribution!

Copy link
Member

@charlesBochet charlesBochet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Weiko Weiko merged commit bfc13b9 into twentyhq:main Oct 10, 2024
6 checks passed
@yadavshubham01
Copy link
Contributor Author

@Weiko Thank you for your feedback

@Weiko
Copy link
Member

Weiko commented Oct 10, 2024

/award 300

Copy link

oss-gg bot commented Oct 10, 2024

Awarding yadavshubham01: 300 points 🕹️ Well done! Check out your new contribution on oss.gg/yadavshubham01

harshit078 pushed a commit to harshit078/twenty that referenced this pull request Oct 14, 2024
…q#7553)

Flush Specific Redis Keys After Upgrade Command :----

Changes Made :----
Updated the FlushCacheCommand to allow for selective flushing of keys
that match the pattern engine:*, rather than flushing the entire cache.
Added a new optional parameter to specify the cache keys pattern.
Ensured that the cache is flushed at the end of the upgrade command.

Code Changes :----
Modified FlushCacheCommand to include a method for flushing keys by
pattern.
Added a new function in CacheStorageService to handle the pattern-based
flushing.

---------

Co-authored-by: Weiko <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add cache-flush step in Twenty upgrade command
3 participants