Wow its incredible how well a WAF works! WAF’fin awesome! It needs some tweaking to reduce costs but it pretty much cleared up all our ddos shite. At least thats what we’re suspecting at this point.
Everything crashed for like two days straight. Even now, we’re still not recovered fully from all the crap we were trying to get things unbroken but we’re close. Crazy amount of extra stress loaded on our servers by endless calls and relentless db queries.
Well it turned out to be either a DDoS attack or some kind of malware. We’ll post more info below as we analyze the logs. Anyway, we’re sure glad thats over!
I had remembered seeing a web application firewall option near the load balancer tools when we were setting the ALB in AWS. I went ahead and enabled that. There were a few standard options I flipped on, added a few choice IPs from the web log and bango!
The problem solved for now. The traffic was back to normal in about 10min.
The quickly learning WAF dispatched the invaders!
The bummer is all the messing with the infrastructure for the entire day and night Looking for problems to blame on our perfectly functioning innocent new redis implementation. What did we do wrong we were whining!!
Well as a by-product of the fiasco, we’re a lot more familiar with Redis. Also, I had no idea Redis could function as relational database as well. We’ll be looking into that shortly!
The lame thing is that we nave been down for so long. The cool thing is we learned!
