From Disruption to Reinvention: DeGore's Commitment to Community Safety
Dear DeGore Community,
We hope this message finds you well. We'd like to thank you for your patience and understanding during this outage. We believe it's crucial to maintain our transparency by providing a comprehensive post-mortem of the incident: what happened, why it happened and the steps we've taken to address it.
The Discord URI Evolution: A Sudden Change
On a seemingly ordinary day, Discord introduced a subtle but impactful change to the way they handle direct image URIs by adding query parameters (starting with ?ex=) to the URI.
This additional information stores the following data alongside the image's URI:
Take a note of the ?ex= and the following query parameters:
- ex: 65db0b0b
- is: 65c8960b
- hm: 0e4f377f38d60924eb7a7ee713b8170443385d36a68fdd7f5d20664a265c052b&
exrefers to the expiry timestamp of the link, in UNIX secondsisrefers to the issue timestamp of the link, in UNIX secondshmrepresents an HMAC signature created by Discord's secret key, using the expiry and issue timestamps as input. It's crucial for the signatures of theisandexparameters, when processed with Discord's key, to precisely correspond to the value specified in thehmparameter; otherwise, an error will occur and the image will not be accessible. Given that Discord exclusively possesses the encryption key required to generate these signatures, it's not feasible to generate your own value for thehmparameter.
What is hot-linking, anyway?
Hot-linking occurs when external websites or applications directly link to resources hosted on Discord's servers, such as images. In essence, it's a form of bandwidth theft, where these external entities consume Discord's resources without contributing to the platform's ecosystem.
This not only poses a potential security risk but also places a considerable strain on the platform's infrastructure, resulting in increased server load and higher operational costs.
By introducing the query parameters (?ex=) to the direct image URIs, Discord aims to mitigate hot-linking, ensuring that users who benefit from Discord's services are the ones consuming its resources. This move not only helps to reduce load on Discord's end, but also fosters a more sustainable and equitable usage of its platform.
The Technical Glitch: Unraveling the URL Conundrum
Discord's alteration meant that the image URLs needed these query parameters for accessibility. However, our URL detection logic, implemented through Regex code, unintentionally stripped away these vital parameters when identifying image links.
The consequence was a series of invalid URLs in our system, leading to an outage that impacted the core functionality of DeGore. Whilst the image links were detected properly, the needed query parameters were missing, resulting in Discord's content delivery network refusing to load the images and consequently detection failing.
Image Detection: Evolved 💪
In the face of this challenge, after thorough investigation and testing, we modified the code at the core of DeGore's image processing systems to detect and process image links, whether or not they contain additional query parameters.
We are also actively developing the HASH PURGER, a feature aimed at re-scanning image links from previous messages in a channel where an action was taken (e.g. image deletion). This will further fortify our system, preventing any missed images, especially certain tricky links, from slipping through the moderation process.
Additionally, we plan to implement this to further improve image blacklisting; when an image is blacklisted from the logging channel, DeGore will re-scan the images sent from previous messages, to look for previous instances of the blacklisted image being sent, even before it was blacklisted.
Looking Ahead
At DeGore, we understand the critical role our AI-powered image moderation solution plays in fostering a safe and positive online community. We sincerely apologize for any disruption caused by the recent outage and assure you that we are continuously working to enhance our systems, ensuring a robust and reliable experience for all users.
Thank you for your ongoing support and we hope you continue to enjoy DeGore's rapidly growing set of features available to protect your community.
Best Regards,
Savvy
CEO & Lead Developer, DeGore
https://savvydev.me