Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Project Idea] FediThreat: Federated Threat Intelligence Service - IP/Email/URL reputation service #1

Open
shleeable opened this issue Feb 20, 2025 · 10 comments
Assignees

Comments

@shleeable
Copy link
Collaborator

shleeable commented Feb 20, 2025

Problem to solve:

Blocklists stopping bad actors running bad instances from communicating with good actors on good instances makes sense.... but once the administrators have completed a good block list... using https://cariad.fedicheck.iftas.org/ or similar and blocked all of the bad instances.

The next step would be good instances from dealing with lower risk like spam, phishing and other kinds of abuse.

The problem I want to solve is spammers wasting moderators time across the fediverse:

A spammer joins instance A, gets kicked
A spammer joins instance B, gets kicked
A spammer joins instance C, gets kicked
A spammer joins instance D, gets kicked
A spammer joins instance E, gets kicked
..... vs
A spammer joins instance A, gets kicked
A spammer joins instance B, gets kicked
A spammer cannot join instance C due to alert.

Solution:

The Fediverse thrives on openness and decentralization, but bad actors exploit this freedom to spread spam, abuse, and malicious content. FediThreat is a shared security service that helps Pixelfed, Mastodon, and other federated platforms assess risks in real time.

How It Works:
An instance can query FediThreat with an IP, email, or URL, and receive a risk score based on collective intelligence and threat data.
Why FediThreat?

✅ Protect Your Community – Proactively detect and mitigate threats before they spread.
✅ Shared Intelligence – Benefit from a federated database of known bad actors.
✅ Privacy-Focused – Designed to respect user data while enhancing security.
✅ Easy Integration – API-first approach for seamless deployment on any instance.

By working together, Fediverse admins can fight abuse, spam, and malicious activity

A spammer shouldn't be able to jump from instance to instance with the same username/email/ip without being flagged.

Informal -

  • "This IP is believed to be from Australia"
  • "This IP is a tor exit node"
  • "This IP is AWS EC2"
  • "This Email is using a disposable/temp email provider"

Critical -

  • "This IP is suspended by 5 other instances - for SPAM/SPAM/SEO/ABUSE/SPAM"
  • "This email is suspended by 3 other instances - for CSAM/ABUSE/ABUSE"
  • "This profile URL is a known security issue - phishing"
  • "This comment URL is known spam - SEO"

would be a good start... these signups could be moved to manual review to catch false positives.

--

FediThreat can add detection over time

EDIT: It is critically important for this project to focus on "TIER 1" reports from Instance admins/mods in the fediverse instead of pulling in second/third hand data from other providers.

As an admin, I will add spammers/abusers what hit my instance and that data should be considered higher quality, than random threat data from third parties for any kind of automation

but third party data can be used for research for real people to make choices.

Reference: mastodon/mastodon#18970

@shleeable shleeable changed the title [Project Idea] FediThreat: A Federated Threat Intelligence Service - IP/Email/URL reputation service [Project Idea] FediThreat: Federated Threat Intelligence Service - IP/Email/URL reputation service Feb 20, 2025
@shleeable
Copy link
Collaborator Author

shleeable commented Feb 20, 2025

We would need to build a frontend to allow trusted admins/mods to log into and submit samples.

Once a server is on the server directory listing... I'd imagine they could be invited to join

@todash-chimes
Copy link

Would this service just be a front end that an admin can use to query a public threat feed and return back data?

Or

Would it maintain its own intel based on various threat feed sources (VT, talos intelligence, etc), and make that available to admins to query?

@shleeable
Copy link
Collaborator Author

shleeable commented Feb 21, 2025

I figure we start simple.. but I could do a little investigation into different kinds of "open source intel" that can we can ingest.

https://github.com/disposable-email-domains/disposable-email-domains also makes sense...

@dansup dansup self-assigned this Feb 21, 2025
@shleeable
Copy link
Collaborator Author

shleeable commented Feb 21, 2025

Most of these are closer to anti-email spam or similar.. but there are alot of overlap.

Spamhaus - Check if an IP is reported for malicious activity.
DeHashed – Search for breached credentials and leaked data.
Hunter.io – Verify email deliverability and domain trustworthiness.
EmailRep.io – Provides a reputation score for email addresses.
AbuseIPDB – Check if an IP or email is associated with abuse.
Scamalytics – Detect fraud and spam email domains.
Disposable Email Domains List – Open-source list of known disposable email services.
AbuseIPDB – Check if an IP is blacklisted for malicious activity.
IPQualityScore – Scans for VPNs, proxies, Tor nodes, and bad reputation IPs.
MaxMind GeoIP – Provides location, ISP, and ASN details for IPs.
Cisco Talos – IP and domain reputation scoring.
Shodan – Query open ports, services, and security risks related to IPs.
VirusTotal – Scan URLs/IPs for malware and suspicious behavior.
Tor Project Exit Node List – Identify if an IP is a Tor exit node.
Google Safe Browsing API – Detect malicious websites and phishing links.
PhishTank – Check if a URL is reported as a phishing site.
URLScan.io – Analyze how a URL behaves (redirects, embedded malware, etc.).
CheckShortURL – Expand shortened URLs to detect hidden phishing or malicious sites.
VirusTotal URL API – Scan links against multiple antivirus databases.
StopForumSpam – Database of known spammers based on IP, email, and username.
Project Honeypot – Detect known spam bots and abusive IPs.
Spamhaus DBL – Detect malicious domains used for spam.
RiskIQ PassiveTotal – Track domain reputation and associated IPs.
Shadowserver – Threat intelligence feeds for compromised networks.
IBM X-Force Exchange – Cyber threat intelligence.
AlienVault OTX – Open threat intelligence database.

@dansup
Copy link
Member

dansup commented Feb 21, 2025

https://github.com/pixelfed/FediThreat

Will be transferring this issue to that repo shortly!

@dansup dansup transferred this issue from pixelfed/pixelfed.foundation Feb 21, 2025
@shleeable
Copy link
Collaborator Author

Data freshness is important.. I figure data from third parties can be cached for 24 hours

but reports of a bad email is likely bad forever. A bad IP is only accurate for a few months.

plus we need to offer people a way to get their username/email/ip removed from the list incase of false alarms.

@jhaar
Copy link

jhaar commented Feb 21, 2025

Perhaps such ThreatIntel feeds could be grouped into categories ("known spammers", "compromised email addresses", "anonymizing networks") and then each Fediverse admin could choose what their risk profile looks like? e.g. some definitely would want to block known spamming IPs - but allow TOR - whereas some would want to block it, etc.

@shleeable
Copy link
Collaborator Author

shleeable commented Feb 23, 2025

Perhaps such ThreatIntel feeds could be grouped into categories ("known spammers", "compromised email addresses", "anonymizing networks") and then each Fediverse admin could choose what their risk profile looks like? e.g. some definitely would want to block known spamming IPs - but allow TOR - whereas some would want to block it, etc.

I think this concept works well... we'd just need to try and get a list of categories completed before 1.0

and we can expand on them over time hopefully.

@Mr-Andersen
Copy link

Since you are aiming for a decentralized "intelligence" exchange, you could use something like TrustNet to prevent bad actors from poisoning everyone's data

@shleeable
Copy link
Collaborator Author

Since you are aiming for a decentralized "intelligence" exchange, you could use something like TrustNet to prevent bad actors from poisoning everyone's data

I'll look into trustnet later.

Thanks for this... for the beginning, reports will only be accepted by manually approved/trusted instances
but we could expand this to every instance can submit reports, and we can calculate trust based on the instances history.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants