Wednesday, February 13, 2008

More useful code for user_defaults.erl

xd on freenode #erlang posted a useful code snippet today (pasted below). This code adds several useful functions to the shell. If you're the original author of this, let me know so I can give you credit!


-module(user_default).
-author('serge hq.idt.net').

-export([help/0,dbgtc/1, dbgon/1, dbgon/2,
dbgadd/1, dbgadd/2, dbgdel/1, dbgdel/2, dbgoff/0, lm/0, mm/0]).

-export([time/1]).

-import(io, [format/1]).

help() ->
shell_default:help(),
format("** user extended commands **~n"),
format("dbgtc(File) -- use dbg:trace_client() to read data from File\n"),
format("dbgon(M) -- enable dbg tracer on all funs in module M\n"),
format("dbgon(M,Fun) -- enable dbg tracer for module M and function F\n"),
format("dbgon(M,File) -- enable dbg tracer for module M and log to File\n"),
format("dbgadd(M) -- enable call tracer for module M\n"),
format("dbgadd(M,F) -- enable call tracer for function M:F\n"),
format("dbgdel(M) -- disable call tracer for module M\n"),
format("dbgdel(M,F) -- disable call tracer for function M:F\n"),
format("dbgoff() -- disable dbg tracer (calls dbg:stop/0)\n"),
format("l() -- load all changed modules\n"),
format("nl() -- load all changed modules on all known nodes\n"),
format("mm() -- list modified modules\n"),
true.

dbgtc(File) ->
Fun = fun({trace,_,call,{M,F,A}}, _) ->
io:format("call: ~w:~w~w~n", [M,F,A]);
({trace,_,return_from,{M,F,A},R}, _) ->
io:format("retn: ~w:~w/~w -> ~w~n", [M,F,A,R]);
(A,B) ->
io:format("~w: ~w~n", [A,B])
end,
dbg:trace_client(file, File, {Fun, []}).

dbgon(Module) ->
case dbg:tracer() of
{ok,_} ->
dbg:p(all,call),
dbg:tpl(Module, [{'_',[],[{return_trace}]}]),
ok;
Else ->
Else
end.

dbgon(Module, Fun) when is_atom(Fun) ->
{ok,_} = dbg:tracer(),
dbg:p(all,call),
dbg:tpl(Module, Fun, [{'_',[],[{return_trace}]}]),
ok;

dbgon(Module, File) when is_list(File) ->
{ok,_} = dbg:tracer(file, dbg:trace_port(file, File)),
dbg:p(all,call),
dbg:tpl(Module, [{'_',[],[{return_trace}]}]),
ok.

dbgadd(Module) ->
dbg:tpl(Module, [{'_',[],[{return_trace}]}]),
ok.

dbgadd(Module, Fun) ->
dbg:tpl(Module, Fun, [{'_',[],[{return_trace}]}]),
ok.

dbgdel(Module) ->
dbg:ctpl(Module),
ok.

dbgdel(Module, Fun) ->
dbg:ctpl(Module, Fun),
ok.

dbgoff() ->
dbg:stop().


lm() ->
[c:l(M) || M <- mm()].

mm() ->
modified_modules().

modified_modules() ->
[M || {M, _} <- code:all_loaded(), module_modified(M) == true].

module_modified(Module) ->
case code:is_loaded(Module) of
{file, preloaded} ->
false;
{file, Path} ->
CompileOpts = proplists:get_value(compile, Module:module_info()),
CompileTime = proplists:get_value(time, CompileOpts),
Src = proplists:get_value(source, CompileOpts),
module_modified(Path, CompileTime, Src);
_ ->
false
end.

module_modified(Path, PrevCompileTime, PrevSrc) ->
case find_module_file(Path) of
false ->
false;
ModPath ->
case beam_lib:chunks(ModPath, ["CInf"]) of
{ok, {_, [{_, CB}]}} ->
CompileOpts = binary_to_term(CB),
CompileTime = proplists:get_value(time, CompileOpts),
Src = proplists:get_value(source, CompileOpts),
not (CompileTime == PrevCompileTime) and (Src == PrevSrc);
_ ->
false
end
end.

find_module_file(Path) ->
case file:read_file_info(Path) of
{ok, _} ->
Path;
_ ->
%% may be the path was changed?
case code:where_is_file(filename:basename(Path)) of
non_existing ->
false;
NewPath ->
NewPath
end
end.


time(F) when is_function(F) ->
S = now(), Res = F(), E = now(), {timer:now_diff(E,S), Res}.

Monday, February 11, 2008

Weird Java exception XML parsing

For the last couple of weeks, I've been unable to start any of our Java-based Thrift services that require XML configuration on my development workstation. They've been dying with the following informative exception:


[java] Caused by: java.lang.UnsupportedOperationException: This parser does not support specification "null" version "null"
[java] at javax.xml.parsers.DocumentBuilderFactory.setSchema(DocumentBuilderFactory.java:489)
[java] at com.amiestreet.service.facebook.FacebookConfig.createDocumentBuilder(FacebookConfig.java:82)


The same code from the same tree and the same classpath works fine on our VM development systems and our production servers.

After a bit of a witch-hunt, with innocent bystanders such as OpenOffice.org killed along the way, I finally traced the problem to a file /usr/lib/jvm/java-1.5.0-sun/jre/lib/ext/xerces.jar. No idea how it got there, but figured I'd contribute to the google results for this problem and save some poor soul some aggravation down the road.

Thursday, January 31, 2008

Alternative Erlang bindings for Thrift

At Amie Street we use Thrift, an open source project from Facebook, to offload various services from our PHP frontend to backends written in various other languages. Last week we released our first backend service written in Erlang, using the facebook-provided bindings.

At the beginning of this week we began to work on our backlog of improvements for this service. In working on these improvements, I realized that the Thrift bindings for Erlang were written as a fairly close translation of the bindings in object oriented languages, to the point of an OOP emulation layer (thrift_oop_server). This resulted in a lot of code that was both confusing for developers to read through and inefficient for the Erlang VM to execute.

Since Erlang is quick to write, I spent the last couple of days rewriting the bindings, and have the new version online in a git repository. Here's the list of notable new features:


  • Improved idiomatic style - the whole thing is more "Erlangy" -- there are less gratuitous processes, and things are named thrift_binary_protocol rather than tBinaryProtocol

  • Improved performance - as a result of ditching the OOP layer, the new bindings are at least 5x faster than the old.

  • More robust code generation - the old version generated broken code from the ThriftTest.thrift file in the repository. The new one passes the tests.

  • Bug fixes - for example, the old version did not properly serialize/unserialize doubles

  • No unexpected crashing processes - the old thrift bindings crashed at the end of every connection when the client disconnected. The new version only crashes when something goes wrong


Hopefully we'll be able to get these new bindings merged upstream in the next couple of weeks. But if you use Erlang and Thrift, please check it out and try it on your project. And if you don't use Erlang and Thrift, it may be worth writing a quick server to see how useful it really is!

Wednesday, January 16, 2008

Reloading all code from the Erlang shell (take two)

So, it turns out that the snippet from earlier doesn't quite work right.

Thanks to #erlang on freenode, I found out that you can add commands to the shell by putting them in the user_default module. So here's the improved version:

In ~/erl/user_default.erl


-module(user_default).

-export([la/0]).

la() ->
Modules = [M || {M, P} <- code:all_loaded(), is_list(P) andalso string:str(P, "amiest") > 0],
[shell_default:l(M) || M <- Modules].


Then compile that module using c(user_default) from within ~/erl.

In your ~/.erlang:

code:load_abs("/home/amiest/erl/user_default").

Tuesday, January 15, 2008

Reloading all code from the Erlang shell

One of the reasons Erlang is pretty cool is that it supports hot code reloading. With OTP (Open Telecom Platform) applications, this can be done through a complicated mechanism involving release files, code_change callbacks to update state, and special upgrade/downgrade scripts.

In development, however, that's a bit of a nuisance. So, how do you simply reload all of the loaded modules after you've recompiled them? Here's a quick solution to paste into your ~/.erlang:


Reload = fun(M) ->
code:purge(M),
code:soft_purge(M),
{module, M} = code:load_file(M),
{ok, M}
end.

ReloadAll =
fun() ->
Modules = [M || {M, P} <- code:all_loaded(), is_list(P) andalso string:str(P, "/home/amiest") > 0],
[Reload(M) || M <- Modules]
end.


Obviously, replace /home/amiest with a directory within which your modules reside. Then, simply run ReloadAll(). from the shell to load the newest versions of your modules.

Wednesday, December 26, 2007

Launching ASDoc with External Tools

While redoing the player, I have been writing a lot of actionscript lately. And by a lot I mean a ton. After I had everything working properly, the time came to go back through and clean it all up. In that process, I came across ASDoc, the documentation generator for actionscript. Like JavaDoc or PHPDocumentor, you mark up your code with multi-line comments, and after running the tool, you get a nice pretty set of HTML pages that make it understandable.

As usual, I dove right in. Marked up all of my methods and member variables and was ready to go. But, ASDoc has quite a few arguments to customize the output and using the shell to run this every time I made a change got old real fast.

Now enter External Tools from the Eclipse framework. If you haven't used external tools in the past, basically it allows you to configure and run mind numbing tasks click and go style. After digging through the flex mailing lists for a bit, I came across this. Problem solved.

But.... I had several projects I wanted to document and I didn't want a million of these external tool configurations all over the place. Another nice feature of External Tools is variable substitution on workspace and project levels. So, here is how to add the external tool to run asdoc on the currently open project:
  1. Click Run-> External Tools -> Open External Tools Dialog
  2. Click on the New Configuration button
  3. Name it Generate ASDoc
  4. For location, enter the path to asdoc. On windows, the default is C:\Program Files\Adobe\Flex Builder 3\sdks\3.0.0\bin\asdoc.exe
  5. In the working directory, enter ${project_loc}
  6. In the arguments input, -source-path . -doc-sources ./com -window-title "${project_name}" -main-title "${project_name}"
  7. Click Apply
After you run "Generate ASDOC", you will get a new folder in your project called asdoc-oputput that contains the fruit of our labor. If you open up index.html, it will look something like this:




Pretty sexy, no?

About the Arguments


Here is a quick explanation of what the arguments in the above instructions do. You can view all of the available arguments here.

-source-path . Sets the working path for asdoc. Why not use ${project_loc} instead of setting the working directory? For some reason, external tools doesn't like spaces in its substitutions (even with proper escaping). Since the default workspace in flex builder contains spaces, this can be quite a pickle. Setting the working directory gets around this (thanks Jimmy!).

-doc-sources ./com Tells ASDoc to generate docs for everything inside ${project_loc}/com/. If you are documenting a Flex Library Project, you can keep the code as is. Note: If you are trying to document a normal Flex Project (ie your code is in ${project_loc}/src/), change the arguments appropriately.

-window-title "${project_name}" -main-title "${project_name}" Sets the HTML title and any other title references to the name of your project.

Monday, December 17, 2007

fcsh-mode for flex work from (x)emacs

A week or so ago I finally bit the bullet and decided I was going to have to learn some Flex/ActionScript so I could be self-sufficient on my latest project, a music game that integrates a Flash frontend with an Erlang backend. Being a Linux user (at work, at least) I downloaded the Flex SDK and was pleased to find that I could compile my Flex application using mxmlc from the command line. However, it was ridiculously slow -- we're talking 25+ seconds to compile a 500 line mxml file.

It turns out the reason for this is that mxmlc is written in Java and has significant startup cost. The solution is to use fcsh, the Flex Compiler Shell, which runs continuously and can incrementally compile your Flex application as you make changes. Using this from my shell reduced compilation time to 1-2 seconds, which is perfectly acceptable.

However, I quickly noticed that fcsh sucks. Most notably, it has no command history. I was typing "compile 1" every 15 seconds and getting annoyed. So I buckled down and wrote an fcsh-mode for Xemacs, reproduced below.

If you want to try it, paste it into a file like ~/.xemacs/fcsh-mode.el and load it using M-x load-file or adding (load-file "~/.xemacs/fcsh-mode.el") to your emacs init file. Then M-x fcsh will launch an fcsh with command history (even cross-session) in a new buffer. I also added an fcsh-repeat-last command which I bind to C-Enter using M-x local-set-key in my editing buffer. I can then just hit control-enter to recompile my flex app in less than a second.

Hope someone finds this useful!


(require 'comint)

(defvar fcsh-mode-map
(let ((fcsh-mode-map (copy-keymap comint-mode-map)))
(define-key fcsh-mode-map [(up)] 'comint-previous-input)
(define-key fcsh-mode-map [(down)] 'comint-next-input)
fcsh-mode-map
)
"Keymap for fcsh major mode")

(defvar fcsh-mode-hook nil
"Functions to run when fcsh mode is actived.")

(defvar fcsh-input-ring-file-name "~/.fcsh_history"
"*When non-nil, file name used to store Fcsh shell history information.")


(defun fcsh-mode ()
"Major mode for running the fcsh, flex compiler shell"
(interactive)
(comint-mode)
(setq major-mode 'fcsh-mode)
(setq mode-name "FCSH")
(use-local-map fcsh-mode-map)

(setq comint-prompt-regexp "^\\(fcsh\\) ")
(setq comint-eol-on-send t)
(setq comint-input-ignoredups t)
(setq comint-scroll-show-maximum-output t)
(setq comint-scroll-to-bottom-on-output t)

;; Some older versions of comint don't have an input ring.
(if (fboundp 'comint-read-input-ring)
(progn
(setq comint-input-ring-file-name fcsh-input-ring-file-name)
(comint-read-input-ring t)
(make-local-variable 'kill-buffer-hook)
(add-hook 'kill-buffer-hook 'comint-write-input-ring)))

(run-hooks 'fcsh-mode-hook))

(defun fcsh ()
"Run an inferior fcsh, with I/O through buffer *fcsh*.
If buffer exists but fcsh process is not running, make new process.
If buffer exists and fcsh process is running, just switch to *fcsh*.
The buffer is put in fcsh-mode.

\(Type \\[describe-mode] in the fcsh buffer for a list of commands.)"
(interactive)
(cond ((not (comint-check-proc "*fcsh*"))
(set-buffer (make-comint "fcsh" "fcsh"))
(fcsh-mode)))
(pop-to-buffer "*fcsh*"))


(defun fcsh-command (cmd)
"Run a command in the fcsh shell"
(interactive "scommand?")
(let ((old-buffer (current-buffer)))
(cond ((not (comint-check-proc "*fcsh*"))
(set-buffer (make-comint "fcsh" "fcsh"))
(fcsh-mode)))
(set-buffer "*fcsh*")
(goto-char (marker-position (process-mark (get-buffer-process (current-buffer)))))
(insert cmd)
(comint-send-input)
(switch-to-buffer old-buffer)))

(defun fcsh-repeat-last ()
"Repeat last command in fcsh"
(interactive)
(let ((old-buffer (current-buffer)))
(cond ((not (comint-check-proc "*fcsh*"))
(set-buffer (make-comint "fcsh" "fcsh"))
(fcsh-mode)))
(set-buffer "*fcsh*")
(goto-char (marker-position (process-mark (get-buffer-process (current-buffer)))))
(call-interactively 'comint-previous-input)
(comint-send-input)
(switch-to-buffer old-buffer)))

Tuesday, December 11, 2007

Concurrency-Oriented Programming

You may have read Todd's earlier post about Erlang. We like it, in part, because it's designed for concurrency. We can write highly concurrent programs without fighting a continuous uphill battle against races and deadlocks. In fact, we barely worry about these things at all. Does this sound like magic to you? It's real, and it's quite relieving. We don't owe this debt to the presence of some clever language feature. On the contrary... Erlang's strength is in the absence of a particularly detrimental feature.

Mutable data.

Yes, we have essentially declared war on the assignment operator. Does this sound bizarre to you? How do we do without it? Well, a long time ago, delete and free() were pretty important. Language designers got rid of them, programmers had a paradigm shift, and we all moved on. Today's assignment operator is evolving into something new... the single-assignment operator. You instantiate a chunk of data, assign an initial value, and promise never to modify it.

The danger of mutability becomes apparent when we investigate the causes of race conditions. You may have noticed that read-only data structures don't need to be locked. Readers don't race with each other. Put a writer on the scene... and locks have to be introduced. Figuring out your locking is probably the most painful and least forgiving part of coding. So, if we nip the problem in the bud, and make all data read-only... then we obviate the need for locking.

The absence of mutable data makes concurrency a trivial problem.

If we can't change our data, how do we pass information between threads? We use a special language construct to do this... message passing. It takes 3 easy steps. Prepare the message you want to send, aim it at another thread, and fire. It is just like opening a socket and sending data over TCP/IP. As long as the language designers provide a sound implementation, you won't have to worry about race conditions.

The next question is, if we can't change our data... how do we program at all? Namely, how do we maintain state? Well, believe it or not, Erlang has a solution for this as well. It's called concurrency-oriented programming, where processes each maintain private state. By making processes extremely lightweight (quick to spawn, completely user-level, having a memory footprint of a few hundred bytes) we can replace objects with processes.

This is the Java code for a thread-safe counter:

class Counter {

private int value;

public Counter(int value) {
this.value = value;
}

public synchronized int get() {
return value;
}

public synchronized void incr() {
value++;
}
}


This is the Erlang code for a thread-safe counter:

-module(counter).
-export([new/1, get/1, incr/1]).

new(Value) ->
spawn(fun() -> loop(Value) end).

get(Counter) ->
Counter ! {self(), get},
receive
{Counter, Value} -> Value
end.

incr(Counter) ->
Counter ! incr,
ok.

loop(Value) ->
receive
{From, get} -> From ! {self(), Value}, loop(Value);
incr -> loop(Value+1)
end.


There you have it. We've translated a stateful, encapsulated object into a stateful, encapsulated process. Notice we haven't used the assignment operator. It took just a little bit more code (note that Erlang provides nice libraries to abstract away the looping and the message passing). Feel free to instantiate as many as you want; processes are about as cheap as objects.

Inexpensive concurrency makes the absence of mutable data a trivial problem.

The benefits here are numerous. It is automatically race-free, and does not use lock-based synchronization. Work can be done asynchronously (a function can essentially return a value before finishing its computation). The process and caller can transparently live on different machines.

This also fully exploits the power of multi-core processors. An Erlang program might have millions of processes (just like a Java program could have millions of objects). Therefore, you will not exhaust your speedup potential until you're using a million-core processor. Whereas, a Java program with 10 threads maxes out on a 10-core processor, and does not speed up when adding more cores.

I haven't even discussed internals, like garbage collection, which leverage immutability immensely.

While OOP provides proper encapsulation and modularity... COP takes us to the next level and helps us embrace our highly-concurrent world. It's time for another paradigm shift.

Updates for 12/11/07

Mostly bug fixes today.
  • Fixed: existing playlist presets in the player and added a few new ones
  • Fixed: The mystery of being logged in as a ghost user
  • Improved: The Twitter service can now take any username (more on that later today)
  • Improved: The library browser in labs now supports tag browsing and browsing the library of all your amie street friends
  • Fixed: lightboxes are back to fixed position. No more losing the lightbox when you scroll on the page.

Newsletter for December 11th, 2007

This week's Newsletter has finished sending. Check it out here. This week features 8 Albums in: Alternative, Pop, Rock, Country, Rap, Folk, Acoustic, Hard Core, Metal, Indie Rock, and Soundtrack.

Dont like email? Get the newsletter by RSS.

Monday, December 10, 2007

Erlang and "Web 2.0"

At first glance, one might not guess that Erlang, a programming language invented in the 1980s by Ericsson (the Swedish phone equipment company) would be a great tool for Web 2.0 development. One of the most important parts of the included library is the "Open Telecom Platform" and error messages remind me of my Apple IIe:


=ERROR REPORT==== 10-Dec-2007::22:12:48 ===
game terminating for reason: {noproc,{gen_server,
call,
[{global,song_picker},
{pick_song,undefined}]}}
** exited: {noproc,{gen_server,call,
[{global,song_picker},{pick_song,undefined}]}} **


Recently, though, the language has been gaining a lot of buzz online, and for good reason. So, a few weeks ago I picked up the book "Programming Erlang: Software for a Concurrent World." I quickly skimmed it in an evening, and passed it along to Jason who did the same. By the end of the week, we both liked it enough to consider writing some Amie Street backend services in it, and the week after that convinced ourselves it was actually a good idea.

So, why is Erlang a good match for web-based services? Here are a few reasons:


  • Functional programming and dynamic typing - with a dynamically typed language, you can start writing code and have working prototypes within minutes. Good functional code can be much more concise than its procedural equivalents, and less lines of code usually means less bugs and shorter development time.

  • Concurrency-oriented programming - in Erlang, pretty much everything is its own process. If you've got 20 users playing a Flash game, or 100 shopping carts outstanding, just give them each their own process. State machines show up all the time in web programming, and a state machine having its own thread and not having to worry about in-DB storage makes the whole thing a lot easier to think about.

  • Fault tolerance - because everything is its own process, one user running into a bug can't cause the whole system to go down. Processes can monitor each other and recover gracefully when errors do occur. For example, in the 2-player Flash game I'm writing, if a user quits his browser, an AI player will automatically take his place for the remainder of the game, and the code to do this is exceedingly simple.

  • Some great standard libraries - best example: mnesia, a built-in database that supports full ACID transactions, disk/ram storage, replication, hash-based fragmentation, and more.

  • thrift support - the Thrift RPC library from Facebook has Erlang bindings, so you can make calls into an Erlang service from PHP, Perl, Ruby, or whatever your frontend happens to be written in.



Using Erlang, we've implemented a prototype "song labeling" game like the Google "image labeler" in less than 1000 lines of code. It stores tags to an mnesia database, speaks directly to the Flash client using XMLSocket, and is damn near impossible to crash. The first working prototype took about 12-16 man-hours of development time between the client (Flex 3) and the server. I'd love to see someone beat that in any other language.

Tuesday, December 04, 2007

Updates for December 4th, 2007

Thursday, November 29, 2007

Linus Torvalds (creator of Linux and the git source code control system) will tell you that if you use Subversion you are an idiot. I won't go that far, but git certainly saved the day this morning.

Around 10am, our sysadmin was performing a small upgrade to our office filer when Something Went Wrong™. Our filer holds a lot of data, including the source code that us engineers work on all day long every day.

If this had happened two months ago when we were using Subversion for source code control, we would have been off to the bar for some 11am drinks with no access to our source code. And we would definitely have been unable to complete the big release we're planning for tonight ( 320 files changed, 6937 insertions(+), 1636 deletions(-) at present count).

With git, however, we simply designated Jonah's virtual machine as the new central repository and went about our business, pushing and fetching our commits from there, with essentially no productivity lost. Below you can see how the "origin" remote got stopped in its tracks, but we just continued developing inside remotes/jonah. When the filer (origin) gets back up, we'll just push there, remove our jonah remote, and live happily ever after.

New Feature: Fan Email Alerts

The much requested email alert option when an artist you're a fan of uploads new songs will be going in in tonights update. You can find this new option on your fan preferences page or when you become of new fan of an artist.

Tuesday, November 27, 2007

Newsletter for November 27th, 2007

A little late, but this weeks newsletter is out.

Read it here

Check out the blog post here.

Update for 11/26/07

Nothing too big last night. Mainly bug fixes. See below.

Additions:
Bug Fixes:
  • REC Alerts turned back on
  • Old friend confirmation page now has info on new system instead of erroring (http://amiestreet.com/friend/confirm/248171)
  • Fixed DB error in newsletter archives
  • View more fans of an artist now staying more up to date and faster
Found a bug? Send us feedback.

Monday, November 19, 2007

Its Back!

After almost a year, we've decided to bring back the Dev Blog. For those of you who don't know, I tried to keep this pretty up to date the first few months of Amie Street. As things went on, I just got too busy.


However, since the time of this last post, we've been making leaps and bounds on the development side of things, quietly trying to improve the service, try/create new technologies to meet our needs, and playing around with new toys. So much has happened in the last nine months, its incredible. We've added 5 of the greatest engineers in the world and got a real office (we've come a long way) .

So, whats the point of this? We'll try and keep some things scheduled regularly. In the next hour or so, we'll push out a fresh set of bug fixes to the production servers. Every Monday and Thursday, we'll post immediately after the update a list of bug fixes and improvements (in this case tomorrow morning). Tuesday and Friday, we'll also post a running list of upcoming bug fixes and improvements for the next update.

Here's a quick run down of what we'll be posting here:
  1. Talking about what new features we're working on
  2. Share some code tricks we've learned (whether thats javascript, PHP, Flex, Java, erlang, or oCaml)
  3. What its like in the office
  4. Our rants on technology, music, and everything in between

In general, the point is to give you a look inside, trade thoughts on music and technology, and build better tools for you all for finding, sharing, buying and selling your music and hopefully meeting some cool people.

Drop us a line and share your two cents. Is this description up to snuff? Are there other things you would like to see here? Should we stop this entirely and just got back to writing code? As always, it's up to you.

Tuesday, March 06, 2007

Barenaked Ladies: New Album. Free. No DRM. Now.

We've been working all night to keep the servers up and running. Right now everything is peaked out. Trying to get a new box up and running as soon as we can.

Sorry for the lack of updates. Was really busy during the overnight.

More updates as soon as I can.

Monday, October 16, 2006

WSJ Article

Finally got this online. Flash Paper PDF Word