Currently we have some performance issues with karrot. It’s causing some of the information to not load correctly.
We have been having increasingly slow performance for some requests, and a change was deployed yesterday which looked like it might help with that, but today we find it’s even slower!
Here is the GitHub issue which goes into the technical details → Retrieving conversation messages is extremely slow for some users · Issue #2369 · karrot-dev/karrot-frontend · GitHub
Here is the change that was deployed → Message list: split queries to avoid slow cases by tiltec · Pull Request #1141 · karrot-dev/karrot-backend · GitHub
Me and @tiltec are currently chatting in https://chat.foodsaving.world in the #karrot-dev channel about it.
If we can’t find out why soon, maybe we’ll revert the change, and go back to more thinking!
I’ve made a few temporary changes to how we’re running it which hopefully should make it usable now, whilst it’ll take a bit longer to resolve it properly.
I’ve disabled the place statistics (which is on the “feedback” page within a place), and that will show a server error message, and show it loading forever, like this:
It seems that particular request is somehow exceedingly slow, then the whole server gets blocked on those slow requests. Not exactly sure, but something like that!
@tiltec or I will update this thread when we progress a bit more!
1 Like
We figured out the root cause of the performance issue and developed a workaround for it → https://github.com/yunity/karrot-backend/pull/1144
The fix seems to be fine on dev.karrot.world. We will apply this change to production later or tomorrow!
A bit more details: yesterday, we didn’t just deploy the change that Nick mentioned in the first post, but also more things that piled up in the last months. One of them was an update to Django, the web framework that Karrot uses. This update contained a small change that should generally improve the performance of database queries. Unfortunately in our special case, it caused the massive slowdowns you experienced today.
1 Like
I just deployed the fix to karrot.world, should be fine now!
1 Like
Yes, looks good and we see the update :-). Thanks a lot!
1 Like
