The Wayback Machine - https://web.archive.org/web/20140415002045/http://www.pxdojo.net:80/

Tuesday, December 17, 2013

Bitcoin Private Key Necromancy

tl;dr: https://github.com/pierce403/keyhunter

~ The Longer Version ~

A few weeks ago, a friend came to me with a problem.  Way back in 2011, he had the great idea to reinstall Windows.  Without thinking too much about it, he installed the new version of Windows, and used the drive for a while.  It was only later that he realized that the drive actually contained a good quantity if bitcoins.  Luckily, he realized there was a chance that the actual data containing the keys may one day be recoverable, and immediately unplugged it and stored it away for safe keeping.

With the price approaching 1000 USD/BTC, he brought the drive to a local bitcoin meetup and asked around.  One guy ran various profession forensics tools against the drive with no luck, and at the end of the night, the drive ended up in my hands.

Discussions of forensic hygiene out out of scope for this particular blog entry, but needless to say, my first step was using dd to pull the raw data off the drive, giving me a 160 gig file on my local filesystem to work with.

Idea #1 BerkeleyDB recovery
So, of course, the original though was "find and pull out the wallet.dat".  My tool of choice for this sort of thing is magicrescue (http://www.itu.dk/people/jobr/magicrescue/).  Magic rescue is typically used to recover images and documents from large blobs of data, for example, damaged filesystems.  Unfortunately for me, there was no BerkeleyDB "recipe file", which is what magicrescue uses to reconstruct files.  With a bit of poking around, I figured out how to write my own custom recipe files, and I was on my way.  Using a hex editor, I checked out the first 16 bytes or so of a normal wallet.dat, and confirmed that it was the same across multiple wallet.dat files that I had lying around.  I ran the magicrescue scan, and came up with no hits.

I read up some more on the format of BerkeleyDB files, kept tweaking my recipie files to support more and more versions of BerekelyDB, and nothing.

Idea #2 Find *Something*
At this point, I started digging around in the middle of my wallet.day files for anything that might be somewhat unique.  The first few things I tried were coming up negative, and then I noticed the string name"1, which was immediately followed by a bitcoin address in the various wallet.dat files I had.  I built the recipe, ran the scan, and got a single hit.  I looked into the output file, and there it was, a bitcoin address.  I looked the address up in blockexplorer, and there it was.  An address with the exact number of coins my friend had guessed was on the drive, and no transactions since 2011.

!!YAY!!

My next thought was that I needed to carve the wallet.dat file out of this chunk of data I had found.  After a bit of futzing around, I noticed that almost directly above the address was a header for a .NET Assembly.  This meant one thing: fragmentation, which was bad news for me. 

Idea #3 Raw Key Extraction
Okay, it was time to finally figure out how to extract the keys directly.  I found various tools for printing out private keys, but everything was outputting this strange 400 byte format, which didn't seem right.  I read up a bit more about how private keys work in bitcoin, and read a ton of code and specs figuring out how they were supposed to be encoded, and "Wallet Import Format".  That let me to this nifty webapp http://gobittest.appspot.com/PrivateKey.

My big break came last night when I realized I could export one of my own empty private keys, and get the raw 32 byte hex from the website.  Once I knew that, I could dig around in the wallet.dat file.  I noticed that there were some interesting bytes that preceded the private key, and noticed that this preceding magic number was in front of all of the private keys in the wallet.  I quickly whipped up a magicrescue recipe, and before I knew it, I had 400 hits.  I wrote up some code to go through the files, translate the 32 byte data into base58 WIF keys, and threw them into a shellscript that ran "bitcoind importprivkey ...", and imported them into a local wallet that I had.  When that was done, I ran "bitcoind getbalance" and there they were :-D  I quickly moved the coins to a safer place, and let my friend know the good news.

The birth of KeyHunter
I figure not everyone wants to dink around with magicrescue, so I wrote up a tool called keyhunter to automatically rip through a large chunks of data, and spit out the base58 Import Address.  The code is here:

https://github.com/pierce403/keyhunter

If it helps you find any of your lost and forgotten coins, I've set up a donation address here:

1YAyBtCwvZqNF9umZTUmfQ6vvLQRTG9qG

Good luck!

Tuesday, June 25, 2013

Locking down your Tor usage

Due to increased interest, I figure I should post about this in a more public area.

Problem statement : "I worry that unintended data is leaking from my machine while I'm using Tor."

The solution : Using iptables to block all data exiting the machine that is not coming from the Tor daemon.

The way that I do this is by creating a simple script in my home directory called "torlock.sh".  This file contains the following lines:

sudo iptables -F
sudo iptables -I OUTPUT -o wlan0 -m owner ! --uid-owner debian-tor -j REJECT

This assumes a few things.

  1. You are on a Linux box with iptables.
  2. The local tor server is running as the user "debian-tor".
  3. You are connected to the internet through the wlan0 interface.
  4. You don't already have complex iptables rules in place.
  5. You are using the standard tor daemon, without vidalia, privoxy, or the browser bundle.

This will work out of the box for anyone on Ubuntu or Debian who installed it via the supported PPAs.

To get this working with the browser bundle, I first set a password for the debian-tor user, made sure the home directory was set to /var/lib/tor , and then installed the browser bundle there.  Then, when I want to run the browser bundle, I first run the ./torlock.sh script, then run "su debian-tor".  At that point, I can connect to anything using tor, and no traffic from my admin user, or even from root, can exit the box.  Any scripts or tools you're using should be run as your regular user, and you are guaranteed that they will only be able to touch the internet through tor.

Wednesday, December 28, 2011

A simple mtgox auto trading bot in python

Just for kicks, I decided to write a tradebot the other day, and thought it might be interesting for others to see just how simple it is to get a basic bot going.  As a quick disclaimer, the bot that we end up with is the bot that I am running right now, but it probably cause you to lose all of your money.  I am assuming that you have git installed, a working python installation, and a reasonably well funded mtgox account.

step 1 : download a good trading API

git clone https://github.com/viorels/mtgox-trader.git

This code is dead simple to use, and makes writing your bot take no time at all, so you can focus more on trading strategies rather than worrying about your POST variables getting to mtgox alright.

step 2 : move defaultsettings.py to settings.py and fill in credentials

step 3 : test out a hello world app

Create a file like hellomtgox.py, and put in these 4 lines of code


from settings import *
balance = exchange.get_balance()
btc_balance = balance['btcs']
print "you have %s bitcoins in mtgox!" % btc_balance


You should then be able to run   python hellomtgox.py   If successful, you know that your connection with mtgox is working correctly, if not, there might be a problem with your settings.py (wrong username/password?)

step 4 : read some example code

Almost all of the code in the directory, other than the settings and the api.py, is example code.  You can read that to get a feel for which methods are available to you, and how you can effectively use them.

step 5 : think of a good trading strategy

This is definitely the hard part, and you are likely going to be modifying your strategy regularly as the market fluctuates.  The strategy that I am using now is pretty simple.  If the market looks pretty volatile  and is very close to the daily high, sell some coins and buy them back when they are 1% cheaper.  If the price is close to the daily low, buy some coins, and sell them back when the market goes up by 1%.  This is a pretty conservative strategy, and mtgox is taking almost half of each win.  This also only works when

  1. the market is volatile, swinging by 4% or so per day
  2. the market is stable around a single point

If the market is trending up or down, this strategy will bet that the market is going to go back to where it was, and if it doesn't, you lose out more and more as the market drifts away from you.

step 6 : write some damn code

The general structure should look like this

import time
from settings import *


while True:
  # gather data
  # analyze data
  # act on data
  time.sleep(10)


And so, without further ado, here is the code that I am currently using to trade with on the strategy as described above.



import time
from settings import *


while True:
  try:
    balance = exchange.get_balance()
    btc_balance = balance['btcs']
    usd_balance = balance['usds']
    print btc_balance+" btc"
    print usd_balance+" usd"


    ticker = exchange.get_ticker()
    last = ticker['last']
    high = ticker['high']
    low  = ticker['low']


    width = high - low
    girth = 1 - (low / high)
    high_scrape = ( high - last ) / width
    low_scrape  = ( last - low ) / width


    print "width: "+str(width)
    print "girth: "+str(girth)
    print "[ %f %f ( %f ) %f %f ]" % (high,high_scrape,last,low_scrape,low)


    orders = exchange.get_orders()
    print str(len(orders))+" orders open"


    if girth < 0.1:
      print "market seems pretty stable, waiting"
      time.sleep(10)
      continue



    if len(orders) > 0:
      print "there is already a pending order"
      time.sleep(10)
      continue


    if high_scrape < 0.1:
      print "high scrape detected!"
      exchange.sell_btc(amount=btc_balance, price=last)
      exchange.buy_btc(amount=btc_balance, price=float(last)*0.991)


    if low_scrape < 0.1:
      print "low scrape detected!"
      exchange.buy_btc(amount=float(usd_balance)*last, price=last)
      exchange.sell_btc(amount=float(usd_balance)*last, price=float(last)*1.0099)


  except:
    print "oh no!!"


  time.sleep(10)

For testing, I put in 100 coins, and have 50 trading at the high end, and 50 trading at the low end (kept as USD by default).  It has been working pretty well as the market was floating around 4, but today I have an order stuck back down at 4.05, so I am hoping that this upward trend comes back down for a bit so I can get that back :-)

Now, you might notice that the only data that I am pulling from to make my decisions is mtgox high, low, and last.  Yes, this is pretty insane.  Any decent bot really should be analyzing a wide range of data sources, like maybe some historical data, market depth (which the mtgox api here give really easy access to), blockchain data, and even data coming out of the blogotwittisphere could prove extremely useful for successful trading.

Good luck, and be careful not to shoot your foot off!