Resolution of "LibraryThing systems under stress"
This is a continuation of the topic LibraryThing systems under stress.
TalkTalk about LibraryThing
Join LibraryThing to post.
1timspalding
Okay, here's a long-ish explanation of the DDoS attack, the downtime, our reaction and the resolution. I'm making a new topic, because the old one is long, and I don't want this to get lost.
Short explanation:
LibraryThing suffered a DDoS attack starting on Sunday—a flood of malicious traffic. We implemented various steps to mitigate the problem and, this morning, put LibraryThing "under" Cloudflare, an industry-standard way to deal with DDoS attacks. The problem is now over and we are dealing with a few lingering issues.
Longer explanation:
On Sunday we started getting huge amounts of traffic and it quickly broke our gateway (load balancer) machine. This was a particularly bad time, both because it was a holiday weekend and because our systems administrator was on vacation out of the country!
When our systems administrator is unreachable, administration falls to the developers, particularly ccatalfo. Chris worked on it over the long weekend, and I helped when I could. Chris deserves our thanks for giving up his Labor Day for this. Our systems administrator also helped when he could, which was generous.
DDoS. The attack was a classic DDoS (Distributed Denial of Service). Basically, a DDoS consists of sending huge numbers of requests to a website, in order to knock it offline. The "distributed" part of DDoS means that the requests come from thousands of computers around the world. These computers are generally ordinary personal computers that bad actors have taken over.
Dealing with DDoS attacks is tricky. Like every website, we occasionally get over-zealous bots scraping us, even using multiple computers. These can generally be resolved by "blocking" the machines (i.e., their IP address). A true DDoS works like that, but no matter how many IPs you block, new ones join the fray. And the scale is tremendous.(1)
"Half Up." On Sunday and Monday we dealt with it by escalating blocks on traffic, eventually blocking virtually all normal traffic, including the bots. But we "whitelisted" IPs from computers belonging to accounts with at least 5 books that had accessed us in the last two weeks. This meant that most regular users could, in theory, use the site. But IPs change and not everyone logs in every two weeks, so it was a stop-gap solution while we found a better one. You may wonder why we didn't open it up to everyone who had used LibraryThing in the last 90 days, or whatever. Unfortunately, whitelisting two weeks' worth of IPs was close to the limit the servers could handle!
We called this situation "half up." Unfortunately it didn't even solve the problem for members. The attack was so large that, even throwing away 95% of requests immediately, didn't solve it, because the rejected requests still took up some of our internet "pipe." The effect was that some members could reach the site, but it was slow or even down.
When we blocked requests, the attack died down. We knew they'd eventually give up. So, after an hour or two, we'd open our servers up again. Unfortunately, within minutes to hours, the attacks would resume.
Jerks are Jerks. During this time we received extortion demands, on Twitter and by email. Confusingly the attackers claimed they were doing a different sort of attack (a SQL injection attack), which they transparently were not, but which we had to investigate. The request was for Bitcoin of course—the currency of choice of criminals everywhere! In time we saw them make the same threats to other companies around the world, large and small, using the exact same words from the same Twitter handles. (To give you a sense of scale, they also threatened and managed to briefly down NCR.com, a division of Citizens Bank!) (1)
On Tuesday we ended up taking LibraryThing offline entirely, to work through options. We could probably have remained half up, but I made the decision to go all the way down. We had a solution in the wings, but we needed time. I wanted to stop fighting fires and get a real solution in place.
The Solution. That solution was to put LibraryThing "behind" one of the major anti-bot systems, such as Cloudflare. Many members suggested we do this, and they were right. In fact, we were in the process of testing Cloudflare before all of this, to deal with traditional bots. But we didn't think we could complete the process without a full-time systems administrator.
It's sad but—plot twist—our current systems administrator is moving on in several weeks. To replace him, but also to "overlap" with him and learn our systems, we had recently hired a new systems administrator, starting in two weeks. When asked, however, this fellow (Ganawa, to be formally introduced later), agreed to come help us early, late Monday. He came through big time, and got us set up on Cloudflare, working through all the weird edge cases that cause problems. This morning we finally brought everything live. He and Chris did great work.
The situation is now stable. Cloudflare is excellent against DDoS attacks. The attacks soon stopped anyway, because there was no point. DNS "propagation" delays means some users are still blocked, but a small and diminishing number. We are working through a few lingering issues, such as a problem with exports.(3)
I want to emphasize that this was an "external" attack. They clogged our traffic up, they didn't "hack" us. LibraryThing members' data was not affected. We have in any case multiple backups, in multiple locations.
The Future. Going forward, LibraryThing will be using Cloudflare. Although not cheap, it's worth it, and has other advantages. With new experience and new tools under their belt, our system administrators will be working on security generally. We can't promise LibraryThing will never be attacked again—that's just how the web works, especially today—but we are in a better position to respond. Chris and I learned something too, and will be learning more. Our non-technical staff will be working on response and communication plans for any future crises—social-media posts, etc.
Thank you to our members who both dealt with an unfortunate situation and sent their support. LibraryThing just turned 18, and as one member remarked, this will be a birthday to remember!
1. As this article lays out, you can apparently buy a DDoS attack online, for as little as $30/day.
2. Members have asked about legal recourse. Alas, they are few. Such attacks are nearly impossible to trace, and generally orchestrated far outside the US. We are reporting it, and that's the best we can do.
3. We shut down the export system, along with a lot of other systems, during the attack. We are bringing it up now.
Short explanation:
LibraryThing suffered a DDoS attack starting on Sunday—a flood of malicious traffic. We implemented various steps to mitigate the problem and, this morning, put LibraryThing "under" Cloudflare, an industry-standard way to deal with DDoS attacks. The problem is now over and we are dealing with a few lingering issues.
Longer explanation:
On Sunday we started getting huge amounts of traffic and it quickly broke our gateway (load balancer) machine. This was a particularly bad time, both because it was a holiday weekend and because our systems administrator was on vacation out of the country!
When our systems administrator is unreachable, administration falls to the developers, particularly ccatalfo. Chris worked on it over the long weekend, and I helped when I could. Chris deserves our thanks for giving up his Labor Day for this. Our systems administrator also helped when he could, which was generous.
DDoS. The attack was a classic DDoS (Distributed Denial of Service). Basically, a DDoS consists of sending huge numbers of requests to a website, in order to knock it offline. The "distributed" part of DDoS means that the requests come from thousands of computers around the world. These computers are generally ordinary personal computers that bad actors have taken over.
Dealing with DDoS attacks is tricky. Like every website, we occasionally get over-zealous bots scraping us, even using multiple computers. These can generally be resolved by "blocking" the machines (i.e., their IP address). A true DDoS works like that, but no matter how many IPs you block, new ones join the fray. And the scale is tremendous.(1)
"Half Up." On Sunday and Monday we dealt with it by escalating blocks on traffic, eventually blocking virtually all normal traffic, including the bots. But we "whitelisted" IPs from computers belonging to accounts with at least 5 books that had accessed us in the last two weeks. This meant that most regular users could, in theory, use the site. But IPs change and not everyone logs in every two weeks, so it was a stop-gap solution while we found a better one. You may wonder why we didn't open it up to everyone who had used LibraryThing in the last 90 days, or whatever. Unfortunately, whitelisting two weeks' worth of IPs was close to the limit the servers could handle!
We called this situation "half up." Unfortunately it didn't even solve the problem for members. The attack was so large that, even throwing away 95% of requests immediately, didn't solve it, because the rejected requests still took up some of our internet "pipe." The effect was that some members could reach the site, but it was slow or even down.
When we blocked requests, the attack died down. We knew they'd eventually give up. So, after an hour or two, we'd open our servers up again. Unfortunately, within minutes to hours, the attacks would resume.
Jerks are Jerks. During this time we received extortion demands, on Twitter and by email. Confusingly the attackers claimed they were doing a different sort of attack (a SQL injection attack), which they transparently were not, but which we had to investigate. The request was for Bitcoin of course—the currency of choice of criminals everywhere! In time we saw them make the same threats to other companies around the world, large and small, using the exact same words from the same Twitter handles. (To give you a sense of scale, they also threatened and managed to briefly down NCR.com, a division of Citizens Bank!) (1)
On Tuesday we ended up taking LibraryThing offline entirely, to work through options. We could probably have remained half up, but I made the decision to go all the way down. We had a solution in the wings, but we needed time. I wanted to stop fighting fires and get a real solution in place.
The Solution. That solution was to put LibraryThing "behind" one of the major anti-bot systems, such as Cloudflare. Many members suggested we do this, and they were right. In fact, we were in the process of testing Cloudflare before all of this, to deal with traditional bots. But we didn't think we could complete the process without a full-time systems administrator.
It's sad but—plot twist—our current systems administrator is moving on in several weeks. To replace him, but also to "overlap" with him and learn our systems, we had recently hired a new systems administrator, starting in two weeks. When asked, however, this fellow (Ganawa, to be formally introduced later), agreed to come help us early, late Monday. He came through big time, and got us set up on Cloudflare, working through all the weird edge cases that cause problems. This morning we finally brought everything live. He and Chris did great work.
The situation is now stable. Cloudflare is excellent against DDoS attacks. The attacks soon stopped anyway, because there was no point. DNS "propagation" delays means some users are still blocked, but a small and diminishing number. We are working through a few lingering issues, such as a problem with exports.(3)
I want to emphasize that this was an "external" attack. They clogged our traffic up, they didn't "hack" us. LibraryThing members' data was not affected. We have in any case multiple backups, in multiple locations.
The Future. Going forward, LibraryThing will be using Cloudflare. Although not cheap, it's worth it, and has other advantages. With new experience and new tools under their belt, our system administrators will be working on security generally. We can't promise LibraryThing will never be attacked again—that's just how the web works, especially today—but we are in a better position to respond. Chris and I learned something too, and will be learning more. Our non-technical staff will be working on response and communication plans for any future crises—social-media posts, etc.
Thank you to our members who both dealt with an unfortunate situation and sent their support. LibraryThing just turned 18, and as one member remarked, this will be a birthday to remember!
1. As this article lays out, you can apparently buy a DDoS attack online, for as little as $30/day.
2. Members have asked about legal recourse. Alas, they are few. Such attacks are nearly impossible to trace, and generally orchestrated far outside the US. We are reporting it, and that's the best we can do.
3. We shut down the export system, along with a lot of other systems, during the attack. We are bringing it up now.
2reconditereader
Chris and Ganawa, so many thanks! The heroes we need.
Thanks also to Tim and all other employees who didn't get a weekend.
Thanks also to Tim and all other employees who didn't get a weekend.
5lilithcat
Tim, I cannot tell you how much I appreciate your openness to members about this. Your explanations are clear, and hurrah to you for not giving in to extortion!
Sucks that this happened on a holiday weekend, so extra kudos to everyone who gave up their holiday, especially Ganawa who came in well before his start date to help.
The way you all have responded to this is one of the many reasons I love this site.
Sucks that this happened on a holiday weekend, so extra kudos to everyone who gave up their holiday, especially Ganawa who came in well before his start date to help.
The way you all have responded to this is one of the many reasons I love this site.
6gwernin
Thanks for the information, Tim. Like a lot of folks, this made me aware of how much I use this wonderful site. (and my 18er is in a few days!)
7SirThomas
Thanks for the information and the successful work. Even if most members have an alternative (reading a book), I am pleased that my favorite site is available again.
8benitastrnad
I nearly went crazy without LT for three days (starting on sunday September 3) but am happy to have LT back. I suspected that it was some kind of attack but I thought it was at our University level - since I log into LT through another site. I am so glad that you have resolved the problem. Like so many others I underestimated how much time I spend on LT. It is my only "social" media outlet and when it wasn't there - life was bereft. I did get some cooking done, but no noticeable increase in the amount of reading I did. I am so glad that LT is back.
9lorax
When should we expect export to be restored? I tested it after reading this post and it spins at "0 books processed" (for tab-delimited export).
10aeww
Many many thanks to Chris, Ganawa, Tim, and the rest of the team that worked through a holiday weekend and got the site back up and running. I hope you all get a comp day and then some to rest and recover.
Thank you very muchas gracias!!!
Thank you very muchas gracias!!!
11Flip_Martian
>1 timspalding: Appreciate the updates and well done to all involved getting a solution in place.
12Keeline
It is a tough situation to be in. I know how stressful ti can be to try to resolve things when your systems are under assault. We have about 40 servers for the company where I am a sysadmin.
One of the products CloudFlare tries to offer is blocking IP addresses associated with VPNs. And, while it is true that bad actors may use VPNs for their attacks, millions of legitimate users have them as well. Thus a blanket approach can be very harmful. I use a VPN and have trouble reaching some sites (ABEbooks.com, Etsy.com, OfficeDepot.com) when the VPN is turned on. Sites (mis)behave differently once they detect traffic from an IP they associate with a VPN. It can be confusing what is going on. It is to the point that any problem I have with my computer my wife attributes to the VPN. :) Sometimes she is correct on this.
That I can reach this talk page suggests that my IP is not being blocked today.
When talking to companies like the ones mentioned above, what I often give is a metaphor along this line:
Perhaps this metaphor can break down but the blocking of an IP because someone at one time did something bad on it or might do so is a reaction too far. It is better to use systems that react to bad traffic, like a WAF (web application firewall) and fail2ban to temporarily block IPs that have made bad requests on a repeated basis. This reacts to a changing situation. Of course, IP spoofing is still a factor that is not easily addressed.
LibraryThing and all of its people have been very reasonable and intelligent so I don't expect a problem but wanted to pass along my experience with other sites in case some CloudFlare sales rep offers a "solution" that includes blocking all VPN IP addresses.
I've had people who know I use LT reach out on Facebook to seek assurance that the site would return to service.
James
One of the products CloudFlare tries to offer is blocking IP addresses associated with VPNs. And, while it is true that bad actors may use VPNs for their attacks, millions of legitimate users have them as well. Thus a blanket approach can be very harmful. I use a VPN and have trouble reaching some sites (ABEbooks.com, Etsy.com, OfficeDepot.com) when the VPN is turned on. Sites (mis)behave differently once they detect traffic from an IP they associate with a VPN. It can be confusing what is going on. It is to the point that any problem I have with my computer my wife attributes to the VPN. :) Sometimes she is correct on this.
That I can reach this talk page suggests that my IP is not being blocked today.
When talking to companies like the ones mentioned above, what I often give is a metaphor along this line:
If the police are told that bank robbers have left the building and headed north on Main Street. The police do not have probable cause to assume that all people and cars on Main Street are bank robbers.
Perhaps this metaphor can break down but the blocking of an IP because someone at one time did something bad on it or might do so is a reaction too far. It is better to use systems that react to bad traffic, like a WAF (web application firewall) and fail2ban to temporarily block IPs that have made bad requests on a repeated basis. This reacts to a changing situation. Of course, IP spoofing is still a factor that is not easily addressed.
LibraryThing and all of its people have been very reasonable and intelligent so I don't expect a problem but wanted to pass along my experience with other sites in case some CloudFlare sales rep offers a "solution" that includes blocking all VPN IP addresses.
I've had people who know I use LT reach out on Facebook to seek assurance that the site would return to service.
James
14cindydavid4
>2 reconditereader: ditto. thanks for all you do and keeping us in the loop. You mention it being expensive; how are you funding this? hopefully the jerks go away but that might be wishful thinking. Sounds like you have things in hand.
15timspalding
>14 cindydavid4:
It's an expense, but small compared to our total server bill, and our largest expense is salary.
It's an expense, but small compared to our total server bill, and our largest expense is salary.
16BookHavenAZ
>1 timspalding: So many thanks to Tim, Ganawa and the rest of the crew who worked their tails off over the holiday!!! And I would like to personally thank Abigail for taking the time to respond to the panicked email I sent before it occurred to me to check Facebook (Sorry, I'm old, FB is not my first line of communication ;) ) when I could not even get the Thing's homepage to load.
Not only do I love LibraryThing, my bookstore's inventory is stored on it, so I rely on it professionally as well. Many moons ago I paid for three accounts (yes, that's how long ago) and two are for the store. I am beyond grateful for the continued functioning of LibraryThing and I intend to make a (regrettably small - I AM a bookseller, after all) donation as soon as possible. I would urge other users to do the same as they are able. This cannot possibly have been an inexpensive battle to fight.
Not only do I love LibraryThing, my bookstore's inventory is stored on it, so I rely on it professionally as well. Many moons ago I paid for three accounts (yes, that's how long ago) and two are for the store. I am beyond grateful for the continued functioning of LibraryThing and I intend to make a (regrettably small - I AM a bookseller, after all) donation as soon as possible. I would urge other users to do the same as they are able. This cannot possibly have been an inexpensive battle to fight.
17Karlstar
>1 timspalding: Many thanks to you and the team for your hard work over the holiday! I'm glad you found a working solution so quickly.
18darius52
I work in IT security so I know how stressful this kind of situation can be but thankfully I've never had to deal with an attack of this scale. Thank you to all of the team members who chipped in. I only recently discovered LibraryThing but I've been digging into it as my new cataloging platform and making improvements where I can so I'm glad that a solution was found and that we are back!
19bibliothecarivs
Thank you!
20RidgewayGirl
Tim, thank you to you, ccatalfo, Ganawa and all LT staff for not only getting this issue solved, but also for keeping us up to date on what was going on and calming us all down. It's much appreciated.
21mckait
Your openness, quick action and overall care for the site and it's members is much appreciated. Many thanks
22Carr_Memorial_Lib
Thanks to everyone at LibraryThing for bringing about a quick resolution to this matter. Especial thanks to the two gentlemen who gave up a holiday weekend to become our heroes.
23chocorua
My reasons to love this site just continue to grow and this incident is just further proof as to why. What a fantastic community of both members and site administrators. Here's to hoping karma works its way to those behind the attack and to hoping LibraryThing will continue to be around for a very long time. Happy Birthday!
24karenb
Thanks for everything, Tim, and especially the clear communication about what's been going on.
Thank you for everyone who worked so hard on solving the problem. And welcome, Ganawa!
Thank you for everyone who worked so hard on solving the problem. And welcome, Ganawa!
25AppTrailMuseum
And Ganawa aces the hands-on practical portion of the job interview!!
26Linkmeister
Well done, everyone,, and thanks for the explanation. I'm sorry it took up your holiday weekend (why can't the crooks take three-day weekends like everyone else?).
27CurrerBell
>13 timspalding: Whoo-hoo! Thank you so much!!! I've always been somewhat conscientious about backup, but after this, I'm going to be REALLY conscientious.
29Narilka
>1 timspalding: Thanks to all the staff for getting this sorted and LT back to normal! Much appreciated.
30mcghol
Wow. Thank you so very much for working so hard to keep this incredible service working so well. I was frustrated not to have access to my library while lesson planning, but I never doubted for a minute that you were on it and would fix it. So grateful to you all.
31bnielsen
What >30 mcghol: said :-)
32Musereader
ah, the half up solution is why i could get to librarything with my phone but not my work computer on one of the days, that really puzzled me at the time, but it makes sense now.
33pollysmith207
I keep getting 500 internal server error. How to resolve
34Taliesien
>33 pollysmith207: Post what you are doing/trying to do when you get the error here -> https://www.librarything.com/topic/353472#8225235
35melannen
Thanks for working so hard to get the site back up! DDoSs seem to be on an uptick lately. And what a first impression for Ganawa.
I would like to request an official emergency news update channel that doesn't require users to have an account on an external site - both Xitter and Instagram are currently actively hostile to non-logged-in users getting any information from their site whatsoever, and Facebook is slightly better but not great, so it was really hard to figure out what was up while the site was down.
I would like to request an official emergency news update channel that doesn't require users to have an account on an external site - both Xitter and Instagram are currently actively hostile to non-logged-in users getting any information from their site whatsoever, and Facebook is slightly better but not great, so it was really hard to figure out what was up while the site was down.
36MarthaJeanne
>35 melannen: Only, I doubt I would know where to go if I couldn't check LT to get the URL.
I also wanted to say that we always try to be welcoming to new staff members, because we know that they will quickly feel like beloved family members. Ganawa has earned that position before we had a chance to welcome and get to know him.
I also wanted to say that we always try to be welcoming to new staff members, because we know that they will quickly feel like beloved family members. Ganawa has earned that position before we had a chance to welcome and get to know him.
37melannen
>36 MarthaJeanne: Well, that's why getting it established when you're *not* in a crisis is a good idea, so users know where to look when you are.
A lot of sites used to have one page hosted off their servers for situations like this, and it was widely visible on the site, and in a situation where the site was very slow or going up and down, it could be put up very visibly as a better place for people to come for news than reloading the site. Nearly everybody switched to Twitter for this, which made sense when Twitter was reliable and widely accessible, but it is neither of those things anymore.
A lot of sites used to have one page hosted off their servers for situations like this, and it was widely visible on the site, and in a situation where the site was very slow or going up and down, it could be put up very visibly as a better place for people to come for news than reloading the site. Nearly everybody switched to Twitter for this, which made sense when Twitter was reliable and widely accessible, but it is neither of those things anymore.
38waltzmn
>35 melannen: I would like to request an official emergency news update channel that doesn't require users to have an account on an external site
I second this, and had been thinking about the same thing. Just some little web site, such as "LibraryThingStatus.com" with one page on which staff could post notes about what is going on. I don't want Elon Musk or Mark Zuckerberg or such benefiting from my visit. And, as melannen said below, the best time to establish such a site is when there is no crisis.
Twitter/X is not a good answer, because if you go to X in a private browser window and look up LibraryThing, it shows the pinned tweet about LibraryThing being free, and a few other old tweets. Maybe it's possible to find newer tweets, but if one is not an experienced Twitter user (and I am not, because... Elon Musk), locating them is almost impossible.
Possibly this should be posted under "recommended site improvements," but... it's not a change to the LibraryThing site itself, so it's not a recommended site improvements. :-)
I second this, and had been thinking about the same thing. Just some little web site, such as "LibraryThingStatus.com" with one page on which staff could post notes about what is going on. I don't want Elon Musk or Mark Zuckerberg or such benefiting from my visit. And, as melannen said below, the best time to establish such a site is when there is no crisis.
Twitter/X is not a good answer, because if you go to X in a private browser window and look up LibraryThing, it shows the pinned tweet about LibraryThing being free, and a few other old tweets. Maybe it's possible to find newer tweets, but if one is not an experienced Twitter user (and I am not, because... Elon Musk), locating them is almost impossible.
Possibly this should be posted under "recommended site improvements," but... it's not a change to the LibraryThing site itself, so it's not a recommended site improvements. :-)
39tymfos
Hi, no big issue for me, but just to let you know (I haven't noticed this mentioned): Posting a review and checking the box to share it to Facebook triggers a Cloudflare warning. It actually did post the link on my FB, but the only text that showed was a Cloudflare waring, so I deleted the post because people would likely not click on it and read it.
Again, I'm not bothered by it -- I can cut-and-paste my text if I really want to put it on Facebook -- but thought you should know, if that hasn't been called to your attention.
Again, I'm not bothered by it -- I can cut-and-paste my text if I really want to put it on Facebook -- but thought you should know, if that hasn't been called to your attention.
40AntonioGallo
>1 timspalding: Ad majora semper! Thank you!
41melannen
>38 waltzmn: As far as I know, currently the only way for users without accounts to see a user's recent tweets is either to have a direct link to the specific tweet (and if it's a thread, you only see the tweet that was directly linked and can't follow the thread) or to hope they happen to come up in the Google preview.
(I think there might still be a couple websites that try to get around this, but they break every week or so these days.)
(I think there might still be a couple websites that try to get around this, but they break every week or so these days.)
42waltzmn
>41 melannen: As far as I know, currently the only way for users without accounts to see a user's recent tweets is either to have a direct link to the specific tweet (and if it's a thread, you only see the tweet that was directly linked and can't follow the thread) or to hope they happen to come up in the Google preview.
I readily defer to your superior knowledge, but you are demonstrating my (and your) point: Twitter/X is not a good place to put information for those who are not members. Ditto Meta's various different methods of gathering private information.
I readily defer to your superior knowledge, but you are demonstrating my (and your) point: Twitter/X is not a good place to put information for those who are not members. Ditto Meta's various different methods of gathering private information.
43Taliesien
I don't have or use any social media accounts but public ones are available to view by most internet users. When the site was down down, I just pulled up the latest SOTT newsletter to get the handles LT uses across various SM and went to nitter.net/librarything to get the scoop. Nitter is how I've accessed X to search for particular author news since X started making it difficult to view content without an account/logging in. It's been about 4 months now and Nitter still works the way the old Twitter did as far as guest viewing content.
ETA: External news & updates sources from the SOTT newsletter:
Facebook -> https://www.facebook.com/librarything/ - Public with overlay disabled but for average users not easily accessible w/o account
X -> https://twitter.com/librarything - not public better to use Nitter.net to access the same account https://nitter.net/librarything
Instagram -> https://www.instagram.com/librarythingofficial/ - Not public
Threads (FB's Twitter/X clone) -> threads.net/librarythingofficial - Public (I removed the @ so I the url would show here, you don't need it in the address bar it will load automatically)
ETA: External news & updates sources from the SOTT newsletter:
Facebook -> https://www.facebook.com/librarything/ - Public with overlay disabled but for average users not easily accessible w/o account
X -> https://twitter.com/librarything - not public better to use Nitter.net to access the same account https://nitter.net/librarything
Instagram -> https://www.instagram.com/librarythingofficial/ - Not public
Threads (FB's Twitter/X clone) -> threads.net/librarythingofficial - Public (I removed the @ so I the url would show here, you don't need it in the address bar it will load automatically)
44gwernin
>38 waltzmn: I like this idea. Simpler is better.
45cindydavid4
I am on fb so I got the messages but I did wonder how many others were frustrated to get on. i have no idea how it will work, but having some sort of auxillary page would be a good idea
46clamairy
>1 timspalding: Many thanks to you and your tireless team for getting things back up to speed. And thank you for keeping us updated (in multiple places) during the ordeal.
47lilithcat
Oddly, I was able to see LT's Facebook page the other day, despite not having an account there, but not today.
48mcghol
A super minor thing and I don't want to burden your already overworked staff. But just in case it's related and you want to know... I "pin" books from my library to boards on pinterest to keep track of student's reading. Today, I can't pin anything from librarything. Librarything is the only site where I have that issue, so I wondered if it was somehow related. Just sharing info for you. No need to respond.
49Nicole_VanK
Thanks for keeping us informed folks. That's way more than most other sites do. (And part of why LT is special for me).
50melannen
>43 Taliesien: I did not know about the Threads account (since I don't get SOTT emailed I didn't have that for reference while the site was down, and it doesn't come up easily on a google search.) That's good to know, I'll keep that bookmarked.
I used to use Nitter but since Twitter messed with their APIs it's been unreliable - sometimes sites show content, sometimes they don't. Glad to see it's working for LT atm.
I used to use Nitter but since Twitter messed with their APIs it's been unreliable - sometimes sites show content, sometimes they don't. Glad to see it's working for LT atm.
51norabelle414
I'm having the same issue as >39 tymfos:, when I post a link to LT to social media it shows a cloudflare warning that says "Attention Required!"
52Taliesien
>39 tymfos: & >51 norabelle414: This is normal if LT has set their security level on Cloudflare to highest threat level "..under attack" which blocks certain traffic/links that are typically exploited. As of yesterday? LT was at this status, not sure about today.
53krazy4katz
>1 timspalding: Thank you so much for all the hard work you and your team put in to resolve this issue. LT is my life when it comes to books. I will never leave you and this is the reason why. Best wishes to all. k4k
54timspalding
>52 Taliesien:
We aren't going to announce changes here, but, yes, we aren't at the highest level. Cloudflare still blocks machines, and requires sign-ins sometimes, but somewhat less than when "under attack."
We aren't going to announce changes here, but, yes, we aren't at the highest level. Cloudflare still blocks machines, and requires sign-ins sometimes, but somewhat less than when "under attack."
55r.orrison
Thank you to the team for the hard work! Kudos for not bowing to the extortion demands - it only encourages them and does nothing to stop a repeat attack.
56PawsforThought
Just chiming in with the rest of the choir to say thank you for all the work you do and for sacrificing your holiday to fight the god fight against the bot army.
I was rather miserable during the days LT was mostly down, particularly as it coincided with my Pinterest feed not functioning - I simply didn't know what to do with myself without my two main time sinks. Thank goodness you're back up!
I was rather miserable during the days LT was mostly down, particularly as it coincided with my Pinterest feed not functioning - I simply didn't know what to do with myself without my two main time sinks. Thank goodness you're back up!
57avrego
Gracias desde España por vuestro esfuerzo. Keep working, you are a very useful tool for book lovers!!!
60pollysmith207
>34 Taliesien: Hi, I am trying to change cover and importing from photo I took of cover and stored on iMac desktop and unable to do this whereas previously I was able to do this with no problems. I now encounter a 500 internal server error each time I try to change book cover with my own photo
61paradoxosalpha
I'm also having a 500 error when I try to upload a venue photo. I'm heading over to the Bug Collectors group to review and report.
62gwernin
trying to post a book link into a facebook group brings up the following warning:
LIBRARYTHING.COM
Attention Required! | Cloudflare
LIBRARYTHING.COM
Attention Required! | Cloudflare
63waltzmn
>62 gwernin: trying to post a book link into a facebook group brings up the following warning:
LIBRARYTHING.COM
Attention Required! | Cloudflare
Note the thread, "Sharing to FB," https://www.librarything.com/topic/353517#n8226943. Probably this should go to that thread, so that you can keep up with that particular aspect of this problem.
LIBRARYTHING.COM
Attention Required! | Cloudflare
Note the thread, "Sharing to FB," https://www.librarything.com/topic/353517#n8226943. Probably this should go to that thread, so that you can keep up with that particular aspect of this problem.
64gilroy
Tim addressed that in >54 timspalding: in answer to >52 Taliesien:
