Rackspace Cloud DNS api

I used to use dyndns to point one of my subdomains to my home PC. That was great, but dyndns has started requiring that you log in every 30 days to continue to renew your free account, which is a massive ballache. Then I remembered that I’m a Racker, Rackspace have cloud DNS for free, and I have python in my utility belt. So this was my response to dyndns. Up yours, guys.


#! /usr/bin/env python
# dnsr

import httplib
import json

USERNAME="username"
APIKEY="apikey"

conn = httplib.HTTPConnection("someipservice.com")
conn.connect()
conn.request("GET", "/")
resp = conn.getresponse()
rText = resp.read()
homeip = rText
conn.close()

authJson = """
{
"auth": {
"RAX-KSKEY:apiKeyCredentials": {
"username": "%s",
"apiKey": "%s"
}
}
}""" %(USERNAME, APIKEY)

conn = httplib.HTTPSConnection("identity.api.rackspacecloud.com")
conn.connect()
conn.request("POST", "/v2.0/tokens", authJson, {"Content-Type": "application/json", "Accept": "application/json"})
resp = conn.getresponse()
rText = resp.read()
rJson = json.loads(rText)
token = rJson["access"]["token"]["id"]
account = rJson["access"]["token"]["tenant"]["id"]
print "Token: %s, Account: %s" %(token, account)
conn.close()

conn = httplib.HTTPSConnection("dns.api.rackspacecloud.com")
conn.connect()
conn.request("GET", "/v1.0/%s/domains" %account, headers={"Content-Type": "application/json", "Accept": "application/json", "X-Auth-Token": token})
resp = conn.getresponse()
rText = resp.read()
rJson = json.loads(rText)
#print json.dumps(rJson, indent=4, separators=(',', ': '))
conn.close()

for dom in rJson["domains"]:
if dom["name"] == "mydomain.com":
domainID = dom["id"]

print "domain id: %s" %domainID

conn.connect()
conn.request("GET", "/v1.0/%s/domains/%s" %(account,domainID), headers={"Content-Type": "application/json", "Accept": "application/json", "X-Auth-Token": token})
resp = conn.getresponse()
rText = resp.read()
rJson = json.loads(rText)
#print json.dumps(rJson, indent=4, separators=(',', ': '))
conn.close()

for subdom in rJson["recordsList"]["records"]:
if subdom["name"] == "subdomain.mydomain.com":
subdomainID = subdom["id"]

print "domain id: %s" %subdomainID

conn.connect()
conn.request("GET", "/v1.0/%s/domains/%s/records/%s" %(account,domainID,subdomainID), headers={"Content-Type": "application/json", "Accept": "application/json", "X-Auth-Token": token})
resp = conn.getresponse()
rText = resp.read()
rJson = json.loads(rText)
print json.dumps(rJson, indent=4, separators=(',', ': '))
conn.close()

dnsip = rJson['data']
print "Home ip: %s" %homeip
print "DNS ip: %s" %dnsip

if dnsip == homeip:
print "IPs are the same. Exiting"
exit(0)

requestJson = """
{
"data": "%s"
}
""" %homeip

conn.connect()
conn.request("PUT", "/v1.0/%s/domains/%s/records/%s" %(account,domainID,subdomainID), requestJson, {"Content-Type": "application/json", "Accept": "application/json", "X-Auth-Token": token, "Content-Length": len(requestJson)})
resp = conn.getresponse()
print resp.status, resp.reason
rText = resp.read()
print rText

Now all I need to do is run a cron job every hour or so, and my IP address will update automatically.

Why I’m glad I picked btrfs

btrfs subvolume snapshot @mysql @mysql-backup

That’s why I’m glad! When I want to do backups, I don’t have to lock a single table. Intstead, I just perform an instant snapshot on my db partition, and the back up the snapshot at my leisure.

I’ve just performed a test of it, and found that it quite happily restores the database without any errors.

Hooray for btrfs.

So Rackspace Happened…

…and all I can say is that I’m loving it.

Ok, so the commute is a bit mental, but that’s OK. It’s not a difficult commute, merely a tube and then a train, and it is certainly is very comfortable because I have a seat all along the way. I’ve spent most of my time listening to the Raising Steam audio book.

But back to the company itself. I realised today why I’ve been enjoying it so much. It’s not because it’s in a great office. It’s not because the food is cheap. It’s all because I’m no longer at the end of some chain or hierarchy.

In Engine I was just the sysadmin at the end of a long chain which included techs, creatives, account handlers, etc. Very few people had any reason to come over and talk to me, which, I felt, meant that very few people would simply wonder over to talk to me. It was quite similar in Transact. No one would really talk to me because I was the IT guy. I had no real connection with anyone in the company as a whole because the only excuse to talk to me was because something was broken, which is a bad way to have any kind of relationship with your co-workers.

But in Rackspace it’s all very different. I’m now part of a team, which is part of a large department, who all rely on each other and work with each other all day every day. I’m no longer at the end of a chain, instead I am (or at least will be when my training is complete) a fully functional working part of the mechanism that keeps the company going.

Yesterday, there I was just sitting at my desk, and someone came over to my desk.

“You’re new aren’t you? On the Linux team?”

“Yup, I’m Matthew, pleased to meet you”

“Taylor”

And so the conversation went on. How nice, I thought, someone spotted I’m the new guy and came over to say hi! As he walked away, I wondered who he was and what he did in the company. Then it dawned on me. The day before, I’d sat through the company meeting, and remembered someone called Taylor had done some talking, so I looked him up.

Holy crap. My first week and the freaking president of a 5000 strong company noticed me, knew that I was new and thought to say hi. No one that high up, in any of my previous jobs, had ever taken the time out of their day to just wonder around and say hi to me.

I think I’ve found my new home.

Always mock Adobe

10:44 < SuperMatt> where are you going to next, fanners?
10:45 < fanboy> Adobe
10:45 < SuperMatt> fanboy: cool... shall I give you my password now, or will you just extract it from the db?
10:46 < fanboy> I actually loled smatty
10:46 < fanboy> :)

Why I love using a key-value store

CREATE TABLE objects (
    id integer NOT NULL UNIQUE PRIMARY KEY,
    type text NOT NULL,
    "timestamp" timestamp DEFAULT now()
);

CREATE TABLE attributes (
    id integer NOT NULL,
    key text NOT NULL,
    value text,
    "timestamp" timestamp DEFAULT now()
);

Two tables, that’s all I need for a database that can hold any data I want without having to worry about updating the schema.

The objects table stores an id and the type of object it is. For instance, if we have a user in the DB, the id could be 1 (I’m admin, so why wouldn’t it be?), and the type can be set to ‘user’.

Now, in the attributes table, we simply store everything like this (ignoring the timestamp in this little table because it’s not relevant yet):

id key value timestamp
1 username matthew
1 realname Matthew Ames
1 admin true
1 password 843787ae13bcdd0302978dfaac329b9

But what if I want to change the schema so that it tracks the login IP? Simply start adding this line:

id key value timestamp
1 loginip 1.2.3.4 2014-01-12 07:58:06.053527

Now the real genius of the key-value store: it automatically keeps an audit history of everything that happens. Next time the user logs in, we simply insert a new line, like this:

id key value timestamp
1 loginip 1.2.3.4 2014-01-12 07:58:06.053527
1 loginip 5.6.7.8 2014-01-12 08:01:29.498947

Remember that we’re keeping a timestamp each time? If we want to see the last loginip, we pick the last date, but if we want to view a few audit we just select all the loginips with the same id.

Now that my DB has been created, I can just leave it as is and never have to worry about it again. If I want to add a new data type, I just add a new row the the objects table with type set to the new type’s name.

Now I’ve gone Key-Value, I don’t think I’m ever going to go back.

Password Hashing

Here’s my plan for password hashing.

First of all, we need a number. Something high, but not too high. I’ve picked 1,000 after doing a few little tests. Not too high that’s it blocks the server from doing anything else for a long time, but high enough that it creates a large time delay for anyone trying to create rainbow tables.

Now, we simply do this:

import hashlib

password = "wuelei3I" # just generated by pwgen for this demo
salt = "xe2eigh1iayai7eey6aiGhah" # randomised per user
globalsalt = "Aenohpae4oom0xooxo8eez0o" # a global salt for extra salty goodness
hashcount = 1000

pwhash = password

for i in range(0, hashcount):
    hashObj = hashlib.sha512()
    hashObj.update(salt + pwhash + globalsalt)
    pwhash = hashObj.hexdigest()

Now here’s the great part, we can boost security at any point. In our DB, we actually store the number of hashes that the user’s password requires to be ‘unlocked.’ If the user’s hashcount is lower than a global hashcount, we allow them to log in and immediately recalculate the hash, bashed on the hash we already had. That is, if the user was only hashed 1,000 times, when they log in, the server will check against the global of say 5,000 and realise that the user needs more hashed for her their security to be brought up to date.

Rather than taking the password and hasing it 5,000 times, we can just take the hash we already have and re-hash it 4,000 times. Why is this important? Well it means we don’t have to wait for the user to log in again to re-hash! We can run a cronjob in the background which checks who needs re-hashing. If we do this nice and slowly, we can guarantee that within a small period of time, everyone will meet the minimum security requirement without overloading the server.

The this I’m going to miss most at Engine is this:

From: Wine Bar

To: The entire company

Subject: RE: BURGER FRIDAY!


*ahem*

THE PORK WRAP IS NOT VEGETARIAN!

Thanks all for your help people.

*headdesk*

Chickens are not asexual

11:17 < Deeps> aren't chickens asexual?
11:17 < SuperMatt> I would totally eat any chickens I rear
11:17 < Vroomie> SuperMatt: I presume I can just go down the road to the local
farm and ask them for an offcut
11:17 < Deeps> disregard
11:17 < Deeps> google shows off my ignorance
11:17 < SuperMatt> I don't know where you got the idea that they're asexual
11:17 < SuperMatt> they will lay asexually
11:18 < Vroomie> that is quite spectacular ignorance, Deeps. Bravo :)
11:18 < Deeps> ah, egg laying doesn't require bonking
11:18 < Deeps> thats what i was thinking
11:18 < SuperMatt> damn, I need to finish my quotes system
11:18 < SuperMatt> and put that in
11:18 < Deeps> (i was thinking about my friend who has a number of hens, but no
roosters, but still gets lots of eggs, and got confused)