Tripled our weekly active users

Logged in for an update on our analytics last week and this happened:

Screenshot_2021-09-06_at_9.22.20_AM.png

We tripled our weekly active user count.

We went from ~200 to 600 weekly actives.

Fantastic 🚀

Also terrifying because with all the analytics we have set up I have no obvious explanation for why this happened.

The first thing I looked at was our ads spend.

We get about 200 new users a week from ads. This does not explain the jump because we spent the same on ad the week before.

Our search engine ranking position jumped from 14th position to 12th over the course of the month, but this cannot explain such a sudden jump in one week.

About 400 people hit our website every day. It was slightly better than the week before, but not 3 times better. It’s not like we got mentioned in the press. Google Analytics didn’t register any new sources of referral traffic.

Screenshot_2021-09-06_at_9.59.02_AM.png

Overall there were no major differences at the top of the funnel. This was existing users tripling their activity.

The most recent feature release happened on the 31st of August. To our great disappointment, the spike started on the 30th so we couldn’t;t pretend that we just rolled out some killer product tweaks

Screenshot_2021-09-06_at_10.00.42_AM.png

The next explanation was that our analytics were just broken. This is always an option when working with data and if forces you to constantly second guess yourself. Luckily we also have logged on Twitter API usage to corroborate that the app was actually being used and there was a spike in posts.

So we tripled out weekly active usage, our analytics are not broken, there has been no change in traffic at the top of the funnel and it’s not because of any product changes we made.

We have exactly no idea what is going on.

Luckily, I have to pleasure of working with Sasha. He dug into the issue and had a look at the actual email that the free trails were being registered to. Then he looked at the individual tweets the increased usage was coming from. The vast majority of them were utter nonsense and contained the hashtag #Mammootty

unknown-1.png

It turns south Mammootty is a south Indian actor and apparently, he just celebrated his 70th birthday. he must have hired some kind of entertainment media agency for the occasion and it looks like they were trying to get the hashtag to trend.

unknown-2.png

They created an army of fake accounts on Twitter and used Chirr App to schedule tweets around the hashtag by taking advantage of our free trial. Twitter suspended most of the accounts by the end of the week and pretty much everything has returned to normal.

It was a huge relief to know that our analytics and system are fine and that the bug was in the real world. A real person set up legitimate Twitter accounts and posted real tweets. We’ve instrumented some stricter validations around signing up with bogus emails but apart from that, nothing broke, so there wasn’t much to fix.

The spike in fake usage is going to tank a lot of our conversion and retention metrics. 52,000 new tweets on trial accounts and not a single conversion aren’t going to bode well for some of our experiments, but at least we know what happened. On the plus side, this whole adventure helped push us over the line and now over 1 million tweets have been published on Chirr App 🤯💥🚀

Screenshot 2021-09-12 at 1.42.50 PM.png
👆Screenshot taken on the 12th of September

 
3
Kudos
 
3
Kudos

Now read this

Conversion Rate Optimisation Only Makes Sense If You Have 1000 Transactions A Month

Four weeks ago I got accepted to CXL institutes’ conversion optimisation mini-degree scholarship. As part of the scholarship, I have to write an essay about what I learn each week. This is my fourth report. A conversion rate is a... Continue →