Journal of Omnifarious

17:18 - Business plan

I have a business idea that I strongly suspect is viable. I'm looking for someone who knows how to write business plans who's willing to help me write one up.

It is both product and service based. I suspect the time from getting the money to buy some hardware and tinker to having a viable product is about 2-3 months. I can see several different avenues the business could grow from the first product.

Current Mood: contemplative

14:36 - zanfur knows how to configure a network

I was poking about on Trabant's internal network and noticing a few interesting things. It is a much better configured wireless network than most I've been on. It's somewhat impressive even.

So, I go ask the staff who does it. One doesn't know and the other tells me a name that sounds familiar. Says he always wears a Utilikilt. I hunt down zanfur's LJ profile and show it to her. "Yep, that's him!".

Well... I have a few things I think could be done better about how this wireless network is set up, but not many. And the level of attention to detail is way, way higher than I've seen on any other coffee shop's wireless network, even at places like Tully's.

It's amusing to encounter people you already know, even indirectly, in random contexts.

Current Mood: impressed

10:57 - My main workstation has died

It died before, when I moved, and that was the power supply. It's died again now, and I suspect this time that it's something more serious like the video card or possibly one or both of the CPUs frying from overheating. The fans all turn on but the BIOS screen never comes up.

I want to replace it. The newer dual core AMD CPUs generate even less heat and are a lot faster. But replacing it with technology that mirrors the level of current I sprung for when I bought what I have now would run me about $3000, and I don't have that and won't have it anytime really soon. :-(

My monitor also needs replacing. It's a pretty nice LCD monitor from 2003, but it has problems. The main problem is that it gets muddy when things are moving or changing quickly because the pixel change lag is fairly high. The pixel change time is 25ms or so which means a maximum effective frame rate of 40 frames a second and anything faster is muddy. It's not even that great for watching DVDs. The secondary problem is that it is very slow (often a minute or more) to recognize that the video card is trying to bring it out of sleep mode. I usually get frustrated and push the signal select button to force it, but it's irritating.

Actually, scratch that, a pixel change rate of 25ms means an effective frame rate of 20 frames per second. This is because a pixel should be at the color it's supposed to be for at least half the time in order for it not to be muddy. 20 frames per second means a possible change in pixel color every 50ms leaving a pixel that takes 25ms to change to be at the new value for 25ms before changing again.

Current Mood: worried

13:18 - I'm going to be giving a talk on IPv6!

I'm going to be giving a talk on IPv6 and how easy it is to set up at LinuxFest Northwest 2008!

I've never done anything like that before, but I feel pretty confident I'll be able to pull it off well. I've been in situations a few times in front of a whole ton of people and haven't done poorly, and this will be an exercise in high geekery and so I should be fine.

I signed up for the bus sponsored by pogo linux, but if anybody is going and feels up for giving me a ride, that would be really good.

Current Mood: excited

09:20 - LJ got rid of basic accounts with no ads

I've thought about the demise of LJ for awhile. It's quite clear now from how the news wasn't mentioned in a news post, how it's being spun as 'making it easier for users to sign up' and various other things that LJ really doesn't care about it's users at all anymore. This was really quite predictable from the moment they started accepting advertising at all.

Brad Fitz, the person who started it all has a nice post in which he makes the most excellent observation that it's the users that create the whole reason people want to visit the site in the first place. This observation and a discussion of LJs legal status made me realize something.

The modern corporate structure is a wholly inadequate means of expressing the values and desires of the stakeholders in an organization where most of the value of that organization is created by what a corporation would think of as "its customers". Basically this legal framework has been shoehorned into serving a purpose it is wholly unsuited for because a corporation has only a very weak incentive to take the interests of all the people who create the stuff that enables its existence into account. Those users have made a huge investment into the site and that investment is almost completely ignored by the modern corporate structure and repeatedly leads to disaster when the corporation makes decisions at odds with its most important investors, the users of the site that it is a caretaker of.

User content sites need something other than a corporation, something where the organization is legally obligated to take the interests of those users into consideration as the most important factor in decisions made by the organization. I'm going to have to think for awhile to see if I can think of a structure that would work. It's tempting to think of some sort of trust or something quasi-governmental. I prefer structures that naturally and with very little oversight or intervention align the interests of all the participants.

Current Mood: contemplative

17:28 - Time change and computers

I woke up this morning and discovered my computer had a really different idea of what time it was from my watch and bedside clock. I didn't know the time change was due and it took me a bit to figure out what was up. At first I figured I just mis-remembered what I saw on the bedroom clock before I wandered in to read my email. :-)

I like running computers that keep their time internally in UTC and simply have a database of silly timezone rules that they apply before displaying the time. That's as it should be. It's irritating to me to realize that most people still set their BIOS clocks to local time.

I think the world ought to run on UTC and we should just keep in mind that people in different parts of the world wake up at different times. :-) And certainly any time stored anywhere ought to be stored in UTC and simply formatted for the right timezone on display.

Current Mood: pleased

20:57 - I've been a bit inspired, but also sick

I've been pretty sick, but in the few productive hours I've had today and over the weekend, I've been working on this project I've been suddenly inspired to do, mostly because of the investigation required for me to write this post on Thrift, D-Bus and RPC.

I've been wanting a self-describing binary data format that's simpler and less ugly than ASN.1. I also wanted one in which an extremely fast parser could be built for fixed-length data structures known at compile time. Either through the use of an IDML (which would be optional) to generate code, or crazier techniques like template meta-programming in C++.

Anyway, I've finished a very preliminary first pass at a parser in Python. I've called the whole concept InBus after D-Bus from which it borrows a lot of ideas.

• Python code for InBus parser
• Documentation of type tag meanings.

There are several things I would change about this parser, and a few features I would like to add.

• Selected data types are currently variable length and I would like to introduce a '<count>' syntax for annotating these types with a length and thereby making them fixed-length. I would like to add this capability to the arbitrary precision integer type, the binary blob type, the string type and the array type.

• It needs a type for time that very explicitly states that the time is represented as a arbitrary-precision integer that encodes an offset (positive or negative) in seconds from some base time in UTC.

• I would sort of like to incorporate Thrift's idea of field tags so data structures could be upgraded in a backwards compatible way.

  My current idea for this is a variant of the tuple type that would require a field tag after every type element in the tuple.

Also, that parser is inefficient. Ideally it would build up the parse as one or more calls to Python's struct.unpack each of which would unpack multiple values. Right now, though struct.unpack is used fairly heavily it only ever (well, my fancy arbitrary precision integer parser not-withstanding) unpacks one element in any call.

Lastly, right now, it expects the type value to be immediately preceded by a type spec. That's a design mistake. The type spec and type value should be handled separately except for the 'variant' type.

This brings me to another couple of features I think would be interesting, but very tricky. It would be nice if 'variant' types could refer to a previously used 'variant' type. Partly for efficiency reasons, and partly for better clarity since one use of the variant type is to record information present in various derived classes of some base class. It would also enable encoding recursive data structures in a saner way. Additionally it might be nice to be able to refer to previously decoded values in some way for data structures that couldn't fit into a strict tree.

On interesting thing, I think you could conceivably use this type tag system to describe IP packet layouts or other binary formats that have existed previously.

Current Mood: contemplative

20:30 - Thoughts on Thrift and D-BUS

I've been thinking about Thrift and D-BUS ever since I made this post: Java and XML, bad tastes that are worse together. And then I followed up with a reference to my paper: why CORBA and other forms of RPC are a bad idea.

Those, of course, sparked a lot of discussion. Questioning Java and XML, the darlings of 'Enterprise' computing, would hardly pass without comment. It provoked even more of a strong reaction than I'd expect I'd get from questioning AGILE methodologies.

One person though ask me a really good question. Why did I think D-BUS and Thrift were better?

I had to think carefully about my enthusiasm for these two things, and I've reached some conclusions...

First, I like D-BUS a lot better. The problem domain involves communications between processes on a single computer. Context switch overhead is a big deal here, but latency isn't. Also, many encoding issues are greatly simplified in this sort of environment, though D-BUS handles those well regardless.

There are several things I like about D-BUS. There is a consistent naming scheme for referring to D-BUS objects and interfaces. Interfaces are generally expected to support an interface for describing the interface. So they're self-describing. There is a heavy emphasis on notifications and events. So much so that frequently the return you get from a function call is not the real return and you're supposed to expect a notification later that whatever you asked for has finished. Very asynchronous. And the format of messages is a self-describing binary format designed to support a reasonable set of datatypes and user-defined compositions (aka structs). There is a reasonably heavy emphasis on data. Notifications are all about getting the right data to the right thing so it can do whatever it needs to do with it. Very few imperative 'you must do X in such and such a way or I will be very upset' sort of messages are passed.

D-BUS is RPCish, but many of the things I really dislike about RPC are either non-issues in D-BUS' environment, things D-BUS handles better than most RPC systems or both.

Thrift is not nearly so well designed. The main positive thing it has is a nice language agnostic method of describing a data structure in a way that allows for clever upgrades of that data structure later without breaking old servers or clients.

Thrift is not inherently self-describing. There isn't even a standard for asking an object implementing Thrift to send you its IDML, whereas D-BUS actively encourages objects to implement a standard interface for this.

Thrift has the concept of async methods that return as soon as they've made it to the transport. But the methods are still very function-call like, and they encourage thinking of imperative 'do this, change your state in this way' sort of messages. I asked on the Thrift channel about an 'idempotent' method modifier for a method that could be implemented via HTTP GET and people were puzzled as to why I'd even want such a thing. The idea of using RPC to implement REST were met with general puzzlement.

Lastly there is no standard way of naming objects in Thrift. And the HTTP transport that comes by default just assumes it will be sending data via POST requests to http://something:port/, not much of a standard for naming the thing you're talking to at all.

So, Thrift as it stands is very poor for implementing random publicly accessible services.

But providing a language independent way to describe data structures is pretty darned nifty. The closest things I know of that are widely deployed on the wild Internet are JSON, ASN.1 and XDR. JSON is ugly and designed for Javascript Ajax clients. ASN.1 is just plain horribly ugly, overdesigned and a bear to parse. Given that ASN.1 comes out of OSI-land and they gave us such glorious gems as X.500, this is really no surprise. (Strangely, it's used for SNMP. What does that tell you about the S in SNMP?) XDR is not very self-describing in the data stream (from what I remember) has a bad name in other ways because it was associated with an awful RPC implementation from Sun, and is missing a map type.

I've been thinking that having web pages for interacting with data that will return a Thrift encoding of the data if passed an application/x-thrift Accept header might be interesting. They could even accept application/x-thrift data in HTTP POST or PUT requests. This would allow you to use URLs to name objects and a fairly efficient, easily parsed protocol encoding for sending the data needed to manipulate them.

Current Mood: contemplative

10:19 - Slastdot post I'm proud of

It is best understood in context. This is a response to an article about Richard Feynman, the Challenger, and engineering. And it is specifically in response to a post in which someone blames capitalism for the problem's Richard Feynman outlines in Feynman's Appendix to the Rogers Commission Report on the Space Shuttle Challenger Accident.

Blaming the shuttle disaster on capitalism is erroneous. I do not necessarily disagree with your assessment in general, but capitalism was not at fault in that particular instance. What was at fault was bureaucrats trying to look good to their superiors and present a positive public image at the cost of real engineering.

I would say that in general is the meta-problem, not capitalism. In its current form in the US capitalism has caused the existence of many large entities that use hierarchical systems of command and control. These hierarchical systems frequently make sub-optimal decisions because individual actors within the system act for their own benefit but against the benefit of the larger system they are a part of. Particularly egregious examples of this can be found, and they tend to be highlighted as aberrations, but they aren't. They are merely extrema of a problem that is widespread.

Bureaucracy in general serves to insulate actors from responsibility for the results of their actions. As I recall we didn't see any of the middle management of NASA held accountable for the disaster they caused by attempting to look good for their superiors and the public. And this failure of accountability is endemic to the kinds of hierarchical systems you see in most bureaucracies.

Current Mood: pleased

11:02 - Random epiphany

I was discussing the fact that the default Junit report is singularly unhelpful in that it basically provides you with this really nice interface in which you can drill down to a particular unit test result. But this, of course, is not what you care about. What you really want is a list of the unit tests that failed with links to a detailed log of the failure.

She observed that the practice of making very pretty and complete but functionally useless software and reports was something she expected from the business side of engineering, not the engineering people themselves. I asked her what drove Java development and she conceded that was, Java is all about 'solving business problems', not engineering problems.

This led me to think about an idea I've had for awhile. I've long felt that chasing money was a fools errand in business. What you ought to do is chase down problems that need solving and figure out how to make money from them. The value you're creating should come first, and translating that value to money should come second. Money is the incidental reward for doing something that helps people, not the object.

In this context then I evaluate Java. Java initially received a lot of attention by geeks because many things about it were kind of new and interesting. It was a really innovative way to deliver client-side stuff. Platform independent with a built in security model that could be enforced relatively easily by the runtime environment. I saw a lot of interesting and innovative applets written.

But that was all trashed by three factors. First Netscape had a horrible implementation that didn't get any better. Secondly Microsoft tried to sabotage things and lastly Sun didn't go open fast enough to short-circuit Microsoft's attempts at sabotage. The client side was left in disarray and little attention was paid to it.

OTOH, a whole bunch of businessy types saw Java as the solution to a number of problems they were observing. It was hard to hire good programmers and the available languages made it very easy for bad programmers to make horrible mistakes that completely ruined applications. It seemed like Java made things safe enough that you could let bad programmers write code and at worst it would work poorly instead of completely trashing your system. This seemed like a business godsend and Java gained a lot of traction on the server side of things as a way to let armies of no-talent developers write decent server side software.

The developers with talent mostly felt that Java was a poor re-implementation of Smalltalk and only grudgingly went along with the whole thing.

Fast forward to now, and you're left with a technology who's development has been driven largely by business concerns and only a smattering of geeky enthusiasm. The interesting things that people do with it seem like solutions to problems created by other 'solutions' instead of like any real solution to a problem the universe outside of Java has.

The only really interesting stuff in Java anymore is mobile code. But using it involves dealing with dependency issues that have solutions designed to allow some stupid business to use some incredibly old version of something forever instead of taking the time to do even the minimal maintenance necessary to keep current.

It's all very depressing. Java seemed so promising in the 90s.

But I think, as a general recipe, solutions designed to solve 'business problems' are doomed to failure, or at least doomed to being stupid and badly designed. Don't solve 'business problems'. Solve your own problems, or your friend's problems or something you're excited and interested in. The market for it can take care of itself later.

Current Mood: contemplative

16:50 - The weird and difficult intersection of marketing and design

Simplicity is highly overrated according to the author of one of my favorite books on user interface design " The Design of Everyday Things".

Basically, the gist of the article is that apparent complexity is how people judge how powerful a device is. If you want people to believe that you have a spectacular newfangled appliance that does all kinds of neat, useful stuff it had better have tons of knobs and displays to prove it.

Of course, when people get the thing home they are then baffled by it all and complain bitterly about how hard the thing is to use. So one strategy would be to make the apparent complexity an illusion so that people will perceive your product in a positive light while still finding it reasonably simple to use once they get it home.

That's a really hard set of factors to balance though, and it makes me wonder about how the disconnect between appearance and reality will be handled. Will people see through a product that falsely advertises complexity while actually being a poor, low-quality product that doesn't really do anything fancy?

Current Mood: confused

16:27 - More Java ranting

Today I needed to find a Java class. Much easier said than done.

Of course, first I had to find out that I needed to find the class. I get a very unhelpful message from ant when it tries to run an 'email' task
     [mail] Failed to initialise MIME mail: javax/mail/MessagingException

After digging through (of all things) the ant source code in order to figure out why it might say such a cryptic thing, I discover that the only possible way it could is if these lines of code fail:

            mailer = (Mailer) ClasspathUtils.newInstance(
                    "org.apache.tools.ant.taskdefs.email.MimeMailer",
                    MailLogger.class.getClassLoader(), Mailer.class);

For the unenlightened, this means that it needs to find a class called 'org.apache.tools.ant.taskdefs.email.MimeMailer' (gotta love the horribly long convoluted class names) and instantiate a copy of it using a no-argument constructor. (And I'm not going to explain what that means because I know many of you aren't programmers.). So, in order to figure out why this isn't happening, I search the web. I find out that it may be because of a missing ant-javamail.jar file in then $ANT_HOME/lib directory. I have one there though, and not only that, it contains the appropriate org/apache/tools/ant/taskdefs/email/MimeMailer.class file.

In order to dig further then, I write myself this Java program:

import org.apache.tools.ant.taskdefs.email.MimeMailer;

public class jm {
    public static void main(String args[]) {
        Object c = new MimeMailer();
    }
};

This class does (in much plainer fashion) what the previous code does. So it should fail and give me the exception that the wonderful ant has been so helpfully hiding from me (even when I turn on verbose, debug, diagnostics and every other 'show me stuff' flag I can find, what kind of person writes code like that?).

So, I track down the various jar files I need in my javac -cp argument in order to compile the thing. Then I run it and get the error I want:

Exception in thread "main" java.lang.NoClassDefFoundError: \
org/apache/tools/ant/taskdefs/email/MimeMailer
        at jm.main(jm.java:5)

Yay! Now, to find the mysterious missing class. It will be in no obvious place, clearly. The wonderful find and locate tools will be of no help whatsoever here because it will be hidden inside one of the ubiquitous jar files lying around.

sudo locate -r '\.jar$' | tr '\n' '\0' | \
xargs -0 sh -c 'for file in "$@"; do \
  if jar -tf "$file" | fgrep MessagingException; then echo "$file"; fi; \
done' fred

This tells me that it's found in mailapi.jar. There are about 50 of them (literally) lying around all over creation. This is because there was never a standard place to install jar files where Java would automatically find them. That might require some kind of *gasp* dependency on something not Java, or something. Of course, running a Java program at all requires depending on something not Java, but Java people are very weird that way and like to pretend they live in a universe that contains nothing but Java. Forget the turtles, it's Java all the way down! Honest!

I finally pick one of the mailapi.jar files to copy into a place where they should be found by ant. *rolls eyes* Which of course is a different place than any of the other 50 locations they're in. The stupidity never ends when Java is involved.

Edit: Well, that didn't work. But at least this time I was able to convince ant to print out an exception for me. But that was only marginally helpful as the exception was this:

javax.mail.NoSuchProviderException: smtp
        at javax.mail.Session.getService(Session.java:768)
        at javax.mail.Session.getTransport(Session.java:708)
        at javax.mail.Session.getTransport(Session.java:651)
        at javax.mail.Session.getTransport(Session.java:631)
        at javax.mail.Session.getTransport(Session.java:686)
        at javax.mail.Transport.send0(Transport.java:166)
        at javax.mail.Transport.send(Transport.java:98)
...etc

I was somehow supposed to mysteriously divine that what that really meant was that I needed com.sun.mail.smtp.SMTPTransport from some random jar file called smtp.jar, which I found by repeating that whole bletcherous find trick up above. Aigh! And I only found that that was the class I needed by searching for the exception on the Internet and finding someone who had gotten farther down in the stack trace because they did have smtp.jar.

In Python this would never have happened for any number of reasons. First, there is a standard place for .py files to go. And they're left as .py files so you don't have to use a stupid tool to find them inside of some mysterious container like .jar. Then, the source code is there so when something throws an exception you can always find out why. Lastly people typically do not hide as much information from you when handling exceptions as was hidden from me by the writers of these Java classes.

The only thing that would've saved me this much work in Java is encyclopedic knowledge of ant internals and the Java mail API. So stupid!

Current Mood: annoyed

21:52 - Mixed political news

From Slashdot: Dodd's Filibuster Threat Stalls Wiretap Bill

So, the bill won't go through just now. The act has been tabled until January, to work out a 'compromise'. *sigh* I hope there isn't any sort of weakening of resolve on the two key issues involved, the immunity being the more important of the two, even though I think the other has more potential for huge long-term damage.

Current Mood: optimistic

14:51 - Warrantless domestic spying, and telecom immunity

Just in case any of you aren't on the ACLU watch list...

Today the senate is going to be looking at a couple of bills dealing with the whole warrantless spying issue.

Now, some of you might think that this just involves traffic analysis (i.e. who called whom, when and for how long) for calls starting or terminating in a foreign country, and so is in the not-so-illegal realm. You would be wrong in thinking this. One should always expect government to abuse any power it has, and this one is no exception. It has recently come to light that the NSA wanted Qwest to give them access to communications switches that dealt primarly in domestic calls. And they not only wanted call traffic data, they wanted the contents of the calls. Discussions surrounding this request included no mentions of any sort of limits confining data collected to international calls.

Additionally, other documents and sources reveal that all this data is being used in run-of-the-mill drug cases, not just for terrorism as we were originally assured.

The bills that are before the Senate right now grant the telecommunications companies immunity to liability for having granted the NSA the access they requested in violation of the law. They also grant the power to the government to continue this behavior indefinitely. They must be stopped.

Please look up your Senators and call them to tell them that you do not want these bills passed. The website I pointed at will give you a helpful script to use. The main issues are granting retroactive immunity and legislatively permitting the behavior to continue.

Current Mood: worried

18:32 - My main workstation lives!

My main workstation was refusing to turn on. It would sort of spin the fans briefly then immediately shut down. I brought it in to a local computer shop and he figured out it was the power supply.

It cost me about $150 to replace it (including labor) but now my workstation works again. I'm happy. My network is now whole.

Current Mood: relieved

15:11 - Houston, we have Internet

I have Internet connectivity again. Now waiting for the flood of email to hit my email server. :-)

Edit 19:33 PST: Unfortunately my main workstation will not start. I press the power button, the fans spin briefly then shut down. Further attempts to mash the power button do nothing. Unplugging it for 20 seconds, then plugging it back in again causes the power button to again cause the fans to spin briefly.

My theory is too much dust. I'm getting a couple of cans of liquid air tomorrow and will have at my computer with them and a vacuum cleaner.

Current Mood: relieved

21:00 - Amazon's new Kindle e-book reader is evil and dangerous

The essay The Future of Reading (A Play in Six Acts) lays it all out.

For those of you who haven't read it, I highly recommend Richard Stallman's A Right to Read.

And for those of you who think that your convenience is worth any price... *shrug* I have nothing to say to you other than that we have no common ground, and in this we are bitter enemies.

Current Mood: worried

10:21 - Bizarre networking issue

Packets appear to be being dropped between the time pcap gets them and the time they're seen by the raw:PREROUTING table. Does anybody have any ideas on why this would happen?

Current Mood: aggravated

15:51 - Uncharitable description of ant

Well, some Java guy was annoyed with 'make' not for any of the many reasons people get annoyed with make, but because every time it compiled a Java file it started up the JVM to run the compiler. Starting the JVM ended up taking 4 or 5 times as much time as the compiler run did, so it was very slow. And even worse, make required something other than the most holy and perfect Java!

And this Java guy, being a Java guy, decided that it couldn't be Java that was the problem in this case, it must be make. So he wrote a build tool in Java called 'ant' that used a programming language based on XML (a bletcherous abomination if ever there was one) and invoked the Java compiler by just calling methods on it from within the same JVM ant was running inside so no JVM startup cost was incurred.

Now this horrible spawn of Java, XML and a declarative language for dependencies married to a procedural language for describing how they are updated is called 'ant'. And it's what Java projects everywhere use by default. It is one of the many reasons Java should be shunned.

Current Mood: annoyed

15:02 - I will be offline for awhile soon, backup MX anyone?

My servers will be offline for nearly a week starting on November 30th and ending on December 6th (hopefully). This means that http://www.omnifarious.org/ will be down, and no mail to omnifarious.org addresses will be delivered.

*sigh* This is the part about moving that I hate almost as much as packing.

Anyway, if anybody who is online and running a mail server is willing to be my backup MX and store messages for the duration I would be very grateful. Especially if you have fairly good spam filtering that always blocks spam before accepting delivery so people who send legitimate mail that's accidentally blocked are given a decent bounce message.

Current Mood: annoyed