Tweepsmanager blog

New feature: Pagination

All pages are now split into groups of 100 users now. The main reason for implementing this was because users with a major following never actually got to use the site. The request timed out before they could load a user table.

It’s not without it’s quirks though.

But hey, it’s still an alpha version. Let’s keep pushing stuff out there and fix the flaws as we go along.

Also, rolling your own pagination system is not as straightforward as one might think.

Edit: A better version is now up

Why reading the api-specs can save you hours

I spent the weekend trying to come up with strategies to safeguard against running out of api calls when following/unfollowing users. After manic sessions where food and breaks seemed as distant as monday morning I had, IMHO, a pretty neat solution set up and working with unobtrusive warnings and disabling of actions when the api-call limit was nearing for following/unfollowing requests

The thing is I forgot to check what’s actually rate-limited by twitter.

In my mind it went like this

Follow/unfollow - Limited

Getting remaining api-calls - Unlimited

Getting lists of users - Limited

When in reality it went like this:

GET account/rate_limit_status Rate Limited? No

POST friendships/destroy Rate Limited? No

POST friendships/create Rate Limited? No

GET users/lookup Rate Limited? Yes

Why didn’t I notice? I had implemented a bug in a line of javascript in a file I forgot about that kept changing the results.

The kicker to the story is that the solution didn’t really fit into the scenario where the user-fetching was the only rate-limiting part. So once again, I threw it out. It still hurt.

Lesson learnt: Read the docs. It takes 5 minutes to figure this out, thats less then hours of work. Figure out how to test javascript, testing the ruby code is not enough.

Doing it right - Throwing stuff away

Building stuff takes a long time when you are learning. Me (@pbjorklund) and @th3dz set out on a mission to learn us some rails by building something that (I) actually wanted use.

Building stuff is fun. It’s even more fun when you don’t have a deadline to meet. This does lead to some interesting priorities though.

“Shipping is everything!”, “Hack, ship!” and similar outcries can be heard everywhere.

So. I spent the last month coding up a storm (well, more like a breeze since it’s on nights and weekends). Then one day I looked at the codebase. It was growing every day and tests that were getting slower by the minute.

At this point I was thinking things like: “We need background jobs”, “Better database design” or “I wonder what a NoSQL solution would do for our project”. (Please note that I’m not good at any of these things. I don’t even consider myself good at ruby/rails/javascript yet!)

Then I noticed another thing, the cucumber tests had not been touched for around a month and were all still green.

Realizing that I had spent a month implementing new back-end stuff that only resulted in making the site slower without adding any new features I decided to throw out everything I had been working on.

The cucumber tests were still green. It was now time for a facepalm.

All I had been doing was coding for the “What if?”. 

What if we want to do NLP in the future to discover things about our users? What if we want to track who quit following you on twitter? What if we need to store everything and make daily calculations! STOP.

Compare this to what we sent out to do: Hack together a tool that let’s you easily manage the people you follow on twitter.

The goal from here: Make improvements, evaluate, throw stuff away and keep doing it right. Good thing that one of the goals was to learn stuff, that means that this has not been a total waste of time. I have so far managed to:

1. Catch bugs just by doing TDD. The first time this happened was enlightening
2. Started to realize what tests are all about and how useful cucumber can be even when not talking to a stakeholder. It’s my version of duck-typing.
3. Hit a point where the database tables needed more indexes
4. Wrestle with gem dependancies (and win)
5. Learnt some JavaScript/jQuery
6. Learnt alot about ruby/rails
7. Use google. A lot.

Summary of the day

So. After around 13 hours we now have a webapp that shows 100 twitter users that you follow which don’t follow back, and 100 users that follow you that you don’t follow.

Somehow it doesn’t feel like much… :)

To much time spent “banging head on keyboard”.

But! Tomorrow is a new glorious day.

Oh the anguish

So far the day has consisted of @th3dz doing css.

I (@pbjorklund) have banged my head against the testsuite. Still don’t really know how to actually test this omniauth authentication so I had to settle for making sure that the User model atleast got some coverage.

When the rage started taking over it was time to fiddle with something that would actually get us closer to launch; the twitter gem instead. And oh what a lovely gem it is.

Im still contemplating hiding the actuall gem away and just expose the methods we need, just to make switching to something newer and shinier easier.

Oh the omnipotent omniauth

So after alot of debugging I have set up omniauth to use twitter for user authentication.

First I tried to get some insight from the RailsRumble blogpost on omniauth and with twitter. This led to a fair bit of googling (since env[rack.auth] has been switched out with env[omniauth.auth]). Seems simple but was a bit confusing at first.

Lesson learned: Read the comments section, the solution was in one of the first comments.

Then I ended up trying to figure out why i got invalid login messages from twitter. Finally I gave up and just rolled with a simple user model to handle everything. To my horror I still saw the exact same errors.

But after banging my head for a while I realised that I forgot to add the provider and uid fields to the model. Woops.

Lesson learned: Write some tests for the user model…

First post!

Me and @th3dz decided to get together for a weekend to try and bang out some sort of rails app.

After getting set up in my kitchen and spilling espresso all over the cables we were ready to go.

Since th3dz decided to use a windows machine for rails development (yeah… let’s see about this..) his setup took a bit longer than mine.

“Is it ruby 1.9.2?” “What’s the newest version of rails? 1.3.0?”.

To be continued!