Cook Computing

Following my discovery of continuations a few days ago I've downloaded and installed DrScheme as a tool for investigating them. Scheme seems to be the most well-known language which supports continuations and DrScheme provides a good environment for learning the language. I've not got much spare time at the moment but I'm trying to fit in at least half an hour a day on this.

As far as continuations are concerned, I've not quite reached the point where they are instinctively obvious but they are beginning to make sense and I can write simple Scheme functions which use them. I'm planning to write up a simple explanation here but the potential problem with this is illustrated by the fact that the "simple" explanations which I've read over the last few days now really do seem straightforward whereas to begin with they were definitely not!

The only Scheme compiler I've come across for .NET is Pinku Surana's HotDog compiler. Judging from the web page this is still a work-in-progress, even though I remember it being one of the language implementations hyped up to illustrate the multi-language paradigm of .NET at the 2000 PDC (I remember attending a breakout session by Pinku).

I very belatedly came across an interesting article today called One Runtime to Bind Them All. I don't know how I missed this a year ago, particularly because there was a Slashdot thread on it. The url gives the game away to some extent but this is an updated version of the article with comments from Erik Meijer, Program Manager for the CLR at Microsoft, to provide a different perspective on some of the points raised. I agree with the gist of the article that the CLR as it currently stands is designed for languages such as C#. As the article describes, there is no support for programming language features such as closures, continuations, and tagged data types. Of course, these can all be implemented in IL but at the cost of a big performance hit. However Meijer does hint we want to keep innovating the runtime to better support non-standard languages and already new features such as generics have been announced for a future release. Maybe IL will be also extended in ways such as prototyped by ILX.

O'Reilly have an introductory article on NAnt (via Meerkat). I've thought about using NAnt with XML-RPC.NET but before investing the time in setting it up I'd like to be sure that I'll be able to run NAnt on Linux using Mono. I did some research this morning and couldn't find a definite answer. If anyone has NAnt running on Linux please let me know.

Speaking of Meerkat, last night I wrote an XML-RPC.NET interface and test client to access the Meerkat XML-RPC implementation. This was to test the work I'm doing on optional struct members. The recipe parameter to meerkat.getItems can now be represented by a struct like this:

[XmlRpcMissingMapping(MappingAction.Ignore)]
struct Recipe
{
  // search criteria
  public XmlRpcInt channel;
  public XmlRpcInt category;
  public XmlRpcInt item;
  public string search;
  public string search_what;
  public string time_period;
  public XmlRpcInt profile;
  public XmlRpcInt mob;
  public string url;
  // display recipes
  public int ids;
  public int descriptions;
  public int categories;
  public int channels;
  public int dates;
  public int dc;
  public XmlRpcInt num_items;
}

The XmlRpcMappingAction attribute specifies that all the members are optional. So only a subset of the search criteria members need be set and the others members which will default to null will not be transmitted in the XML-RPC request. Similarly with the num_items member. The other display recipe items are value types and will default to zero if not explicitly set, just the behavior we want. So calls are as flexible as the Meerkat API allows:

Recipe recipe = new Recipe();
recipe.channel = 5209;
recipe.search = "/Java|p2p/";
recipe.search_what = "dc:subject";
recipe.time_period = "30DAY";
recipe.descriptions = 1;
recipe.dc = 1;
recipe.num_items = 3;
Result[] results = meerkat.GetItems(recipe);

Recipe recipe = new Recipe();
recipe.category = 63;
recipe.descriptions = 1;
Result[] results = meerkat.GetItems(recipe);

For handling the results from meerkat.getItems this structure can be used:

[XmlRpcMissingMapping(MappingAction.Ignore)]
struct Result
{
  [XmlRpcMissingMapping(MappingAction.Error)]
  public string title;
  [XmlRpcMissingMapping(MappingAction.Error)]
  public string link;
  public string description;
  public string dc_creator;
  public string dc_subject;
  public string dc_publisher;
  public string dc_date;
  public string dc_format;
  public string dc_language;
  public string dc_rights;
}

Again the default for the structure is to ignore missing members but because we always expect title and link we can override their MappingAction settings individually so that an exception is thrown if these members are missing from any of the structs in a response.

I'll include the interface and associated types in the XML-RPC.NET FAQ when I put out the next release. It feels good to have made this change. Meerkat was one of the first XML-RPC APIs I tested the library with and it has always bugged me that an accurate .NET representation was not possible.

Finally, one other change required to use Meerkat is that its PHP XML-RPC implementation does not like any indentation or newline whitespace between elements in the XML-RPC request. It returns garbage in this situation even though the request is valid. I had to modify the XML-RPC.NET code to work around this and so I'll probably enhance the XmlRpcClientProtocol class to make indentation, etc, configurable.

John Lam has stumbled across a class photo from a Developmentor Optimizing DCOM course held in August 1997. It brings back some memories for me. I'm the guy with the beard and blue t-shirt just behind John at the center of the picture. After a week working in Milpitas I'd driven down the coast over a weekend to attend the course. I'd already done a lot of work with COM at this point but I still learnt a lot during this week. It was hard work but I came away with a much deeper understanding of DCOM.

Strange to think that at height of my interest in DCOM, design work had already started work on its nemesis. Jump forward three years to the 2000 PDC in Orlando and people were running around exuberantly declaring "COM is dead!".

Pinku Surana has described why the lack of tagged datatypes in .NET makes it difficult for dynamically typed languages such as Scheme to perform as well as languages like C#. A couple of days ago Dejan Jelovic posted to the Advanced .NET list on the poor performance of tailcalls in .NET. This will also hinder languages such as Scheme:

Something seems to be wrong with the way tailcall is implemented in the .NET runtime. Instead of being faster than a regular call, it's roughly 70 percent slower.

Tailcalls occur where the last step in a method, say Foo, is a call to another method, say Bar, such that Foo simply returns the result of Bar. The significant point is that once the second method has been called we are no longer interested in the stack frame of the first method, so instead of extending the stack with another frame for method Bar, the frame for Foo can be discarded and replaced by Bars stack frame. This means that we dont have to worry about stack oveflow. In general the two methods can be different but the recursive case where a method calls itself is of particular interest.

Many Scheme programs rely heavily on recursion instead of the looping constructs used in languages such as C# and Java, and it is for this reason that tail call optimization is important in avoiding stack overflow; hence Dejans concern. His blog describes the code he used to demonstrate the performance hit of tailcall.

I wrote a tailcall recursive method for generating the factorial of a number and compiled it with both the Microsoft and Mono compilers:

// call with n = number for which factorial required, a = 1
static long Factorial(long n, long a)
{
  if (n == 0)
    return a;
  return Factorial(n - 1, n * a);
}

Neither compiler generates the tail. prefix instruction which would turn the recursive call into a tailcall:

  IL_0011:  tail.
  IL_0013:  call int64 clasas::Factorial(int64, int64)
  IL_0018:  ret
} // end of method clasas::Factorial

I guess this is an optimization which may be implemented later on. As an aside, gcc supports tailcall optimization to some extent.

Regarding the performance of tailcall in .NET, Jim Hogg of Microsoft replied that the JIT team has plans to improve it.

Postscript: while investigating tailcalls I came across continuations. These look very interesting, not something you come across as a C++/C# developer which is probably why I'm finding it difficult to get my head round them. But this is precisely why you need to explore other programming languages: to see software development from a different perspective so you can then work in your usual development environment in a more imaginative way.

NUnit has a nice feature for specifying that test fixtures or individual tests should be ignored: the Ignore attribute. For example:

[Test]
[Ignore("code for handling different cultures not implemented")]
public void testXmlRpcDouble_ForeignCulture()
{
  CultureInfo currentCulture = Thread.CurrentThread.CurrentCulture;
  XmlDocument xdoc;
  try
  {
    Thread.CurrentThread.CurrentCulture = new CultureInfo("fr-BE");
    XmlRpcDouble xsd = new XmlRpcDouble(543.21);
    Console.WriteLine(xsd.ToString());
    xdoc = Utils.Serialize(
      "SerializeTest.testXmlRpcDouble_ForeignCulture", 
      new XmlRpcDouble(543.21), 
      Encoding.UTF8, MappingAction.Ignore);
  }
  catch(Exception)
  {
    throw;
  }
  finally
  {
    Thread.CurrentThread.CurrentCulture = currentCulture;
  }
  Type parsedType, parsedArrayType;
  object obj = Utils.Parse(xdoc, null, MappingAction.Error, 
    out parsedType, out parsedArrayType);
  Assertion.AssertEquals(543.21, obj);
}

Its very convenient when you think of something that needs testing but which cannot be tested yet for some reason. A bit like making a note in your project todo list except in this case you won't be able to ignore or lose the note: each time you run your tests in the NUnit gui the ignored tests and the overall progress/success bar are marked in yellow instead of the usual green for sucess. This is much better than commenting out code in a test harness where it is likely to be forgotten.

Working on the new code for handling optional struct members in XML-RPC.NET Ive been following a strict unit testing approach of coding tests then implementing the code to pass them. This process highlighted the inadequacies of my existing unit tests. I realized that the tests should not be just concerned with the external interface of a component but should also be much finer grained, for example testing functionality implemented in private methods.

The problem with this is how do you give the test harness, NUnit in my case, access to the private methods. Im currently using #ifdef DEBUG to add public visibility to the required methods in a debug build. This is not ideal because there may be significant differences between the release and debug builds. When I get the time Ill add another build which only affects method visibility and so minimizes any differences. But for now Im concentrating on adding tests retrospectively to any areas which are affected by the optional struct member changes.

In the day job weve been discussing testing recently and Ive noticed that unit testing is mostly seen as something which is performed after the code is written. No wonder the unit testing actually performed is often very inadequate. Once youve written the code, writing and running unit tests are likely to be seen as unnecessary burdens, particularly when there are pressures to announce you are code complete and ready for integration testing with other components.

On the other hand, if implementation is driven by tests you get an immediate benefit from the small incremental effort of implementing them, each piece of code being checked as you implement it and the tests as whole providing continuous regression testing. This also removes the need for the separate unit testing phase at the end of implementation which is all to easily skipped.

The main problem with the test-driven approach is that it is often difficult to write tests where external dependencies are involved. I think the key here is to treat testing as a major driving force at the design stage, always choosing the design approach which allows individual pieces of functionality to be tested as they are implemented, using stubs to emulate external dependencies. Testing should not be a separate section tagged on to the end of the design document, almost as an afterthought, but a concern running through the whole document.

None of this is new but it can be difficult to go from vague approval of the concept of test-driven development to actually doing it. Im just beginning to work through some of the issues in something which actually can make a big difference to the way developers work.

Martin Baulig has posted to the Mono list:

After almost half a year of hacking, we finally have a first public release of the Mono Debugger :-) I spent almost the whole week fixing bugs, missed most of my classes and stabilized and tested the whole thing on FreeBSD on Linux, but now finally everything looks fine ....

His email has all the details.

Mike Amundsen has launched a .NET weblogging service: EraBlog.NET (via the Scobleizer Weblog). I've not had to time to check out the site yet but I did notice that the url for the XML-RPC API when viewed in a browser looks suspiciously like an auto-documentation page generated by XML-RPC.NET. The most recent versions of XML-RPC.NET generate much more informative, and much nicer looking, documentation based on types and descriptive text specified via attributes.

In fact once I extend auto-documentation to handle optional struct members you will be able to describe an XML-RPC interface very well using a .NET interface. Who needs WSDL?

[Speaking of optional struct members I've nearly finished the implementation.]

I wasted some time last night trying to use the .NET WebProxy class to route XML-RPC calls through ProxyTrace. I'd been playing around with some code LLoyd Dupont sent me which configured an instance of HttpChannel for XML-RPC programmatically without using a config file (the example in the XML-RPC.NET FAQ uses a config file):

ListDictionary prop = new ListDictionary();
prop.Add("port", 1971);
HttpChannel channel = new HttpChannel(prop, null,
  new XmlRpcServerFormatterSinkProvider(null, null));
ChannelServices.RegisterChannel(channel);
RemotingConfiguration.RegisterWellKnownServiceType(
  typeof(Server), "myApp/anURI", 
  WellKnownObjectMode.Singleton);

I thought it would make debugging very easy if the client and server were in the same executable, using ProxyTrace so you could watch the XML-RPC requests and responses. However I couldn't get the WebProxy instance to work. It was being ignored. I tried placing the server class in a separate AppDomain before discovering that WebProxy wouldn't work for any clients connecting to servers running on the same machine. Google pointed me to the the answer on Mindreef.blog:

... the System.Net.WebProxy class that comes with the .NET Framework will not proxy HTTP requests to localhost

The solution is to use the machine's hostname or IP address when setting the Url property of XmlRpcClientProtocol and not localhost.

The moral: when you have a problem, never waste time trying to fix it before searching for the answer on Google.

The full code for the combined XML-RPC client and server is below. By the way, an XML-RPC version of SOAPscope would make a nice project for someone.

using System;
using System.Net;
using System.Collections.Specialized;
using System.Runtime.Remoting.Channels;
using System.Runtime.Remoting.Channels.Http;
using System.Runtime.Remoting;
using CookComputing.XmlRpc;
 
interface IHello
{
  [XmlRpcMethod("hello.getGreeting")]
  string getGreeting(string name); 
}
 
public class Server : MarshalByRefObject, IHello
{
  public string getGreeting(string name)
  {
    return "Hello " + name;
  }
 
  public static void Run() 
  { 
    ListDictionary prop = new ListDictionary();
    prop.Add("port", 1971);
    HttpChannel channel = new HttpChannel(prop, null,
      new XmlRpcServerFormatterSinkProvider(null, null));
    ChannelServices.RegisterChannel(channel);
    RemotingConfiguration.RegisterWellKnownServiceType(
      typeof(Server), "myApp/anURI", 
      WellKnownObjectMode.Singleton);
  } 
}
 
public class Client
{
  public static void Run()
  {
    IHello hello = (IHello)XmlRpcProxyGen.Create(typeof(IHello));
    (hello as XmlRpcClientProtocol).Url 
      = "http://netherton:1971/myApp/anURI";
    (hello as XmlRpcClientProtocol).Proxy 
      = new WebProxy("http://localhost:8080");
    while (true)
    {
      Console.Write("Enter name: ");
      string s = Console.ReadLine();
      if (s == "")
        break;
      Console.WriteLine(hello.getGreeting(s)); 
    }
  }
}
 
public class _
{
  static void Main()
  {
    Server.Run();
    Client.Run();
  }
}

Dmitry Jemerov has released a .NET-based aggregator called Syndirella (via Scripting News). It is implemented in C# and uses XML-RPC.NET. I've downloaded it and it works well.

Syndirella has a good implementation of RSS feed auto-discovery and this is where Dmitry uses XML-RPC:

My current goal is to be able to discover automatically the feeds for all sites which I regularly read (and which do have feeds). And neither Kuro5hin.org nor Joel on Software have feeds which can be discovered by links. So I need to fall back to the last option in Mark Pilgrim's autodiscovery scenario - search Syndic8.com. And this requires XML-RPC.

I've done some more thinking on the problem of handling optional struct members in XML-RPC.NET and I've come to the conclusion that although nullable types work well for simple value types such as int and boolean, they become confusing for more complex reference types such as arrays and structs.

For example, in the following struct what type could be used to represent the fact the Numbers member is optional?

struct A
{
  public int[] Numbers;
}

So we do need an attribute to indicate optional members which leaves the problem discussed earlier: how do we indicate that an optional value type is missing. For example, if a response returns an XML-RPC struct and we are using the following struct to represent it, if after deserialization member Optional is zero, we don't know whether this value was actually passed in the XML-RPC struct.

[XmlRpcMissingMapping(MappingAction.Ignore)]
struct B
{
  public int Optional;
}

A solution is to provide nullable types for the XML-RPC types which are mapped onto .NET value types. The following table illustrates which .NET types would be used to map XML-RPC struct member values where the members are either expected or optional. The serialization/deserialization code uses the attribute to determine whether an exception should be thrown if an expected member is missing and the application code can check the reference types for null to determine whether an optional member has been supplied.

Struct B would now look like this:

[XmlRpcMissingMapping(MappingAction.Ignore)]
struct B
{
  public XmlRpcInt Optional;
}

Client code can then test for member Optional being null:

if (retStruct.Optional != null)
  Console.WriteLine("The response contained member Optional");

I don't like it - its too complicated - but I think the advantage of being able to represent all XML-RPC structs accurately, rather than using hashtables, outweighs the complexity. Anyway the latter approach is always available using XmlRpcStruct.

While doing some prototyping on how to handle optional struct members in XML-RPC.NET I realized that it is possible to define methods in C# which differ only by return type. This is a feature of the CLR which most languages don't support and C# is the same...except for the definition of operator conversion methods. For example:

class MyInteger
{
  ....
  public static implicit operator Int32(MyInteger mi) 
  {
    return mi.ToInt32;
  }
  public static implicit operator Single(MyInteger mi) 
  {
    return mi.ToSingle();
  }
  .... 
}

In the resulting IL these two methods have the following signatures:

public static  int32  op_Implicit(class XmlRpcInt mi)
public static  float32  op_Implicit(class XmlRpcInt mi)

I've just uploaded v0.7.1 of XML-RPC.NET which is now released under the MIT X11 license. All the .cs files have been changed to reflect this.

The release also contains a fix in the server-side .NET remoting code: if the XML-RPC method name was different to the corresponding .NET method name, an erroneous exception was thrown.

Sam Ruby asks:

Does XML RPC permit multiple members of a struct to have the same name?

The XML-RPC spec is silent on this issue but the authors of Programming Web Services with XML-RPC deduce:

The only way duplicate names might be feasible would be if the order of members in an array were significant. Because members are unordered, we may conclude that duplicate names are not permitted, because otherwise there is no deterministic means of interpretation for a struct with a duplicate name.

From a practical point of view we have to assume that duplicates are not allowed because most XML-RPC implementations handle structs using a dictionary or hashtable type of collection.

The book also claims that struct member names must be ASCII but I can't find that requirement in the spec. You are restricted to ASCII in string values but not in struct member names? The ASCII/Unicode issue in XML-RPC really does need sorting out! (Simon Fell agrees.)

Version 0.18 of Mono has been released. I've noticed that the debugger under development is in the CVS tree here but I haven't attempted to build it yet. I really need to do that before I make another attempt to run XML-RPC.NET under Mono.

I installed RedHat 8.0 in a VMWare session over the weekend but found that something was soaking up a huge amount of CPU which made everything very slow. However RedHat 8.0 is not supported by VMWare yet so I may need to stick with 7.3 for the time being.

Following on from the previous entry about handling missing struct members in XML-RPC.NET, I coded a prototype using a new XmlRpcMissingMappingAttribute class yesterday. However I'm still not all that pleased with this solution. It gets a bit messy when you want to define an interface representing an XML-RPC end-point where the interface is to be used by both the server and client.

Say a method on the server returns a struct containing two integer members called Required and Optional. You could write a struct like this in C# to represent the XML-RPC struct:

struct Sample
{
  int Required;
  [XmlRpcMissingMember(MappingAction.Ignore)]
  int Optional;
}

When serializing an instance of this struct in a response, if the value of Optional is zero the server cannot determine whether to include a corresponding member in the XML-RPC struct, i.e. a similar problem to that faced by the client as described in the earlier entry.

An alternative approach is to use data types which can be null, for example like Java's Integer, Boolean, etc, classes. I found a couple of archived list messages from other developers looking for something similar: Steve Ballard and Jeroen Frijters (Jeroen is doing some very interesting work on a Java VM for .NET).

Jeroen's message mentions SqlInt32. This and its associated types are value types which have an IsNull property (different to the Java types mentioned above which are reference types). So, the missing mapping problem could be solved by using these types where a member in an XML-RPC struct is optional. The client or server application code could then call IsNull on each of these member to determine whether they were present in the XML-RPC struct.

Going down this route I'd probably provide specific types for XML-RPC.NET, for example XmlRpcInt32, because using types whose name is prefixed with "Sql" does jar a bit. The Mono project has implemented the SQL types so I can use their code as the basis for the new XML-RPC types.

I just need to think through how to handle structs. I have to say that these attempts to support XML-RPC in a statically typed languge does make me look longingly at more dynamically typed languages. This post from Pinku Surana (via Managed Space) was interesting reading about a major deficiency of the CLR for supporting dynamically typed languages; and slightly depressing from a language diversity point of view:

As it stands now, dynamically typed languages run pathetically slow on .NET and JVM. The question is whether C#/Java is so horrible that one would willingly take a significant performance hit to write Scheme/Lisp/Smalltalk? I'm getting used to Java/C#, so I would answer no.

Currently the XML-RPC.NET home page says:

For the time being, XML-RPC.NET is released under the GNU LGPL licence.

Its turns out that at least one user is having difficulties with this license because of corporate policy and so I've been doing some research into using a difference license which doesn't have the copyleft properties of GPL and LGPL.

The MONO FAQ has a section on licensing which explains they are using the MIT X11 license for their class libraries. This is compatible with the BSD license. On the basis that I'd like as many people as possible to use XML-RPC.NET the MIT X11 license seems a good choice. If I was planning to make some money out of some software then I would probably go with GPL which would allow free non-commercial use but retain control over how the library was used commercially. But this has never been a goal of the XML-RPC.NET project.

Drew Marsh has come across a change in XML-RPC.NET 0.7.0 which I should have documented: when mapping the members of an XML-RPC struct to the members of the corresponding .NET struct type, an exception is now thrown if any of the public members of the .NET struct do not have corresponding values in the XML-RPC struct. Previously missing members were ignored and the members of the .NET struct remained set to their default value.

I made this change because I didn't like the case where it was not possible to determine if a .NET struct member had been set to its default value from the corresponding member in the XML-RPC struct, or if the member was missing from the XML-RPC struct and the member in the .NET struct had been left with its default value.

It is perfectly valid for XML-RPC structs to be flexible as to which members they contain. The word "struct" was perhaps the wrong one to use for XML-RPC with its connotations from C and C++. Maybe something like "table" would have been better. To handle situations where the members of a struct are not fixed I implemented the XmlRpcStruct type which is basically a Hashtable with some extra type safety.

However using the XmlRpcStruct type in an interface does not convey much about what the corresponding XML-RPC type may contain and so where possible it is preferable to use a .NET struct type. The interface is easier to use, and from a service point of view you can document individual members using XmlRpcMemberAttribute which results in the documentation appearing in the service's automatically generated web page.

I don't like things which can fail silently in a non-benign way and so I'd like to retain the new default behaviour: if a member is missing from the XML-RPC struct and so can't be mapped to the corresponding member in the .NET struct, an exception is thrown. However for situations where the implementor is unconcerned about the possible confusion described above, I'd like to add a new attribute which can be applied at to the interface/class level, the struct as a whole, or individual members, and which can be used to suppress the missing member mapping exception Drew pointed me to the MissingMappingAction property of the System.Data.Common.DataAdaptor class as the model for this, the difference being that using an attribute provides more flexibility as to where it can be applied.

Any comments?

Ted has replied to my earlier post. He makes some good points although they are essentially predictions about how the .NET world may evolve. I'm actually more of a .NET developer than a Java developer and I'm writing more from a position of frustration as I see Java being promoted around me because it has much better multi-platform support. The future may well turn out as Ted describes but from the point of view of making investment decisions now there is too much uncertaintly about .NET on anything other than Windows platforms. Maybe Sun made a mistake but the Java environment is out there now and the historical business perspective is not all that relevant when people around me want to implement our software on both Linux and Windows (but not Sun machines!).

While on the subject of marketing fluff, I seem to remember Steve Jobs in this week's MacWorld keynote proudly claiming there are now over 10,000 applications for Mac OSX. Its strange how whenever a new unix-like platform is released there are always over 10,000 third-party applications running on it almost immediately. Is there somewhere you can download the source to as many old creaky Unix apps as you need to reach this amazing number?

In a response to the first batch of Carlos Perez' 101 Reasons Why Java is Better than .NET Ted Neward writes:

This promises to be interesting and potentially dangerous. His very first point, "1. Run on multiple platforms from the smallest devices to the largest mainframes", a simple restatement of WORA, isn't a differentiator--.NET runs on multiple platforms (Linux, Mac OS/X) and devices (Windows CE) just as Java does. He's going to have to do better in subsequent posts.

Come on, Ted, you can do better than that. Your blog readers aren't a bunch of PHBs who'll get taken into by that sort of marketing fluff. From the point of view of developing real applications .NET only runs on 32-bit Windows (64-bit Windows 2003 Server is not going to include it).

Although an open source Linux implementation is under development and, patent restrictions permitting, may be a compatible platform in the distant future, I haven't heard anything about Microsoft approving this and providing some process for verifying that implementations are compatible.

All you can run on MacOS/X is Rotor, which is a research/educational implementation which does not contain the full range of libraries available under Windows. Even with this restriction the Rotor licence does not allow you to write commercial applications.

If you had a large sum of money to invest in a product which had to run on various platforms, would you go with Java or .NET? Java may have its flaws but at least you would stand a chance of implementing something. The big difference between the two environments is that Sun is at least promoting Java as a serious multi-platform environment. I don't think you can say Microsoft are doing the same.

Drew Marsh emailed me about some discrepancies between the Blogger and Radio impementations of the Blogger API. It seems that in some cases Blogger uses mixed case and Radio all lower case, for example in struct members.

XML-RPC.NET is case-sensitive with respect to method and struct member names, which I believe is the correct behaviour: there is nothing in the XML-RPC spec to indicate otherwise. Has anyone seen anything to contradict this?

While on the topic of XML-RPC, Dave Winer recently wrote:

Basically, I was (and will) say that we've reached a plateau in our journey of technology. We've got all the basic ingredients for the Two-Way-Web in place. The back-end systems are maturing, with lots of choice. The XML-RPC interfaces are also maturing and (more important) standardizing. We went a lot further with the ManilaRPC interface, and the Edit This Page button in Manila, which is a great idea, still hasn't appeared in other mass market CMSes -- but I have no doubts the interfaces will deepen, and the convenience for users will increase, and the content systems will do more of the work.

Unfortunately before XML-RPC can hold this sort of position in the scheme of things the protocol needs to mature and allow strings containing the full range of Unicode characters[1]. A protocol allowing only 7-bit ASCII characters in strings is a throwback to last century and has no place in today's WWW.

UPDATE: Drew describes the problem fully here

[1] XML-RPC.NET does and always will support full Unicode in strings. If the spec is never changed then at least implementors can create their own de facto standard.

I watched the MacWorld keynote yesterday. Ive read plenty about Steve Jobs but never seen him in action before, although a postage-sized stamp QuickTime image probably did not manage to convey all his supposed charisma. Anyway, it was a very slick presentation, not something Bill Gates can do very well.

Interesting that they are using the KDE browser code. I suppose they couldnt go with Ghecko, regardless of technical issues, because they'd have been making changes to benefit AOL. I think he said something like "We love Open Source at one point. Easy to say when you make most of your money from hardware but they are certainly going all out to woo the developer community. For example, the XFree86-based X11 server looks very cool indeed.

Ive been thinking of buying a Powerbook and I was waiting for MacWorld to see if prices would drop. Looks like my choices have widened considerably. The cost of the 17" model is offputting and Id have to wait about two months. It might also be too big for a laptop, not least because you're going to be a sitting target for muggers walking around with an extra large laptop bag advertising whats inside. But that screen is very compelling...until I think of the cost again.

So it really comes down to the 12.1" and 15.2" models. In the UK there is a 500 difference between them which swings the balance in favour of the 12.1" (comparing the US and UK prices it seems that the 15.2" price has been set at an unfavourable exchange rate to the other models and should be nearly 100 cheaper). At the current price the 15.2" model should have lots more memory and built-in WiFi. But 1024x768 on the 12.1" does seem a bit on the small side these days.

In a comment on John Lams developer co-operative proposal Sam Gentile writes:

Dude, think different is wonderful but what is going to do for your bottom line? I mean, this is all a wonderful theoritical excerise but how are you going to make money from the Mac or use it for your career? I'm not trying to be a thorn but I am curious as to your thoughts.

I agree with the gist of Johns reply, that seeing things from a different perspective is very valuable. Im not sure we even need ulterior motives such as becoming a better programmer. Surely the developers amongst us who spend their evenings and weekends hacking on code or hardware arent really doing it to further their career or development skills. We do it for the intellectual satisfaction it provides. For example I recently implemented an indexed btree class in C#. That came about only because some of David McCuskers writings on btrees had piqued my curiosity, particularly regarding the possibility of implementing them with array-like indexed access. I think I got more intellectual satisfaction out of writing that code than I did from learning how to write server-side controls in .NET. Similarly, I spent some time over Christmas reading up on the various Linux file systems, not because of anything to do with work but because Id been toying with the idea of implementing a file system along the lines of Microsofts Structured Storage and when googling for information I just followed the links that I found interesting.

If you only concentrate on what satisfies your immediate career needs, youll be living in a box with tinted windows. Youll not only see everything in the box in a single shade of colour but worse than that youll be missing out on a world of other interesting and intellectually entertaining software ideas. So spend your evenings and weekends on something different from how you earn a living. It will be much more fun and youll incidentally end up a better developer, a Renaissance developer instead of an MSCE.

I've just uploaded version 0.7.0 of XML-RPC.NET. Its available here.

The changes/fixes in this release are:

• error reporting of parsing errors using parse stack
• support for async proxy method generation
• contiuning work on auto-generated documentation
• params keywords used in XmlRpcClientProtocol.Invoke
• server method can return void (return empty string in XML-RPC response)
• proxy method can return void (return value in XML-RPC response discarded)
• deserializer throws exception if an XML-RPC struct is missing one or more expected members
• fixed irritating warning when compiling XmlRpcStruct
• add version of BeginInvoke taking correct params as per docs
• Close always called on WebResponse
• fixed usage of XmlRpcClientProtocol Proxy property when used in VS designer (Drew Marsh)
• fixed handling of response without Content-Length during async calls (Dmitry Jemerov)
• fixed case when zero-length string in default string value is passed as <value/> (Drew Marsh)

With the addition of of async method generation in XmlRpcProxyGen the feature is set is pretty much complete now. The remaining work is mostly associated with error reporting and automatically generated documentation paged. There are also a few items to add to the FAQ.

I've got the whole of today to release the next version of XML-RPC.NET. I'm by myself so no distractions or excuses. There are a large number of changes in this release so I need to spend some time on final testing. I use NUnit for testing stuff like the serialization code but for features like the generation of async methods in XmlRpcProxyGen, auto-documentation, and error reporting, I run manual tests.

A company called Compoze produces Harmony for Exchange. It's a Java API which uses CDO/DCOM with Exchange 5.5 and, more interestingly, WebDAV for Exchange 2000:

Harmony for Microsoft Exchange 2000 uses WebDAV technology to connect to the Exchange 2000 server. WebDAV offers superior performance for Exchange 2000 users over CDO. Installed on a Java Virtual Machine, Application Server or Servlet Engine, Harmony for Microsoft Exchange 2000 brings the mail, calendar, task, contact, journal, public folders and Global Address List information from the Exchange 2000 server to a web browser or mobile device and store any changes to that information back on the server.

The WebDAV-related products I've mentioned here recently are by no means the only WebDAV products out there, just those Ive seen that are specifically targetted at Exchange 2000. For details of more generally WebDAV-related products and projects the WebDAV Resources page is well worth a browse.

Unfortunately my hosting provider doesn't have half a clue (or maybe wants to minimise disk usage):

Like every self-respecting geek blogger, I spent my New Years holiday crunching the numbers from my 2002 access logs. I only have logs starting in February; before then I was using another hosting provider that didnt provide raw access logs. Note to all geek bloggers: if your hosting provider does not provide raw access logs, dump them immediately and switch to Cornerhost (which I use) or some other provider with half a clue. Youll thank me one day.

Mark Pilgrim analyzes his 2002 server logs.

I can only get logging through use of a Server Side Include (SSI) file to exec a Perl script. The SSI file must have a .shtml extension, which means until now Ive only been able to get logging for http://www.cookcomputing.com/ which defaults to index.shtml.

However Ive devised a method of logging access to individual archive pages. Each of these pages now contains a script element which has its src attribute set to a file called log.shtml. The file is empty except for the exec comment line which invokes the Perl script to update the log. To ensure that log.shtml is retrieved on every request, thereby updating the log, instead of the client using a cached copy held either by itself or a proxy server, the Perl script touches log.shtml and invalidates any cached instances.

The big drawback with this technique is that I dont see the real referrer because the referrer for log.shtml is the individual archive page. But for the first time I can see individual pages being accessed, which will have to do until I decide upon another provider.

I wrote here recently about Outlook Web Access and its use of WebDAV. Today I discovered another application which uses WebDAV with Exchange: Ximian Connector. This is an extension to Ximian Evolution which enables Linux/Unix-based users to access the full functionality of Exchange rather than the more limited functionality available through the IMAP protocol. From the FAQ page:

Q: What features of Microsoft Exchange are supported?
A: Ximian Connector enables Ximian Evolution to function as a Microsoft Exchange 2000 client by communicating with Microsoft Exchange 2000 servers through the WebDAV protocol. End user data, including email boxes, address books, calendars, and task lists can be stored directly on the Exchange Server, and group data, including free/busy information for group scheduling, is available as well. New features in Ximian Connector 1.2 include public folders, access control and delegation, and direct resource booking.

But you still have to pay Microsoft for each user (on top of the licensing fee for Connector):

Q: Do I need a Microsoft Client Access License (CAL) to use Ximian Evolution on Microsoft Exchange?
A: Yes.

Slashdot has a thread on The State of GNU/Linux in 2002: It was Good. From a personal point of view Linux started to impinge on my development work, even on the "day job". After many years of working in a Windows-only environment I was actually running Linux boxes last year in the office, not yet developing code on them but writing LDAP code to connect to OpenLDAP.

The first time I telnetted into a Linux box at work last year it felt strangely reassuring, a homecoming after years of working in the foreign "black-box" world of Windows. How many frustrations over the years with DCOM and MAPI could have been avoided if I could have rebuilt them from the source with my own diagnostic code added? All the time I spent banging my head against the wall could have been used to improve the software, not on finding ways to work around the problems.

Outside of the day job I'm beginning to use Linux more. I'm currently writing this on Mozilla running on a Linux VMware session which I tend to keep running most of the time now and I use gFTP for my website maintenance. My recent hesitant first steps with Perl were made in the same environment. I've even spent some time experimentally porting some of my C# code to Mono running on Linux.

I've not experienced any particular Linux epiphany but I've come to realise what an amazing achievement it is. The fact that it has been achieved by "us", the developers, is astounding. One of the important lessons of 2003 was how the corporate environment has a tendency to untrustworthy behaviour[1] and needs counter-balancing forces to keep it in check. For the development world Linux is such a force, and even die-hard Windows developers should be thankful for that. From a developers point of view Microsoft would be a very different company if Linux and the open-source movement had never existed (but its maybe not quite yet Bedford Falls as opposed to Pottersville[2]).

So I guess one of my resolutions for this year is to get more involved with Linux. I first need to become much more familiar with the environment, learning the ins and out of building and configuring packages, building the kernel, etc. Then I'd like to contribute to Mono. A first step towards this would be getting XML-RPC.NET to run on Mono. As I found out over Christmas this would require some fixes to the Mono libraries which would make it a good place to start contributing.

[1] Read this with or without a touch of irony according to your political point of view.

[2] Its hard to believe but I had never watched Its a Wonderful Life before this Christmas. The film has an unusual history in that it accidentally went out of copyright. The corporate world saw an opportunity to make some money and insult the intelligence of the tv viewing public by colorizing the film, thereby creating new rights on the colorized version. So if you see the colorized version you know that the tv station is paying money to screen a travesty of the film when they could be showing the original for free.

Ben Trott has written a piece on Simple and Powerful Text Formatting in Movable Type which is very relevant to my MTCodeDiv plugin:

But there are problems with the global filtering approach that we hope to solve in the next version of Movable Type.

• Authors have to edit their templates in order to add these filtering options. Not only is this beyond the scope of understanding for some users, but it can be incredibly tedious to edit every template.
• Since filters are set on a template level (in the tags), they are not applied when previewing an entry. This makes preview fairly useless for specially-formatted entries.
• Because templates are shared on a per-blog basis and are not specific to authors, all authors must use the same formatting style.

Looks like I will be upgrading to the next version of Movable Type a bit quicker than last time.

First of my New Year resolutions accomplished. For a while I've wanted to be able to be able to type code and XML into Movable Type entries without having to worry about converting spaces, less-than and greater-than characters to their respective entites. So I upgraded to Movable Type 2.5 1 to get support for plugins and wrote a plugin which looks for the <CODE> tag in entries and processes any enclosed text:

<CODE>
  .... code or XML here
</CODE>

For consistency with existing code samples and XML the output looks like this:

<div class="code">
  .... converted code or XML here
</div>

I didn't do much research into whether there are existing plugins to do this because I've never written anything other than "Hello World" in Perl and this was a good opportunity to write something useful, always the most fruitful way of learning a language.

Its not an ideal solution because you don't see the effect of the plugin on the entry preview page and it doesn't handle certain corner cases but its much quicker than converting the text by hand.

Testing new MTCodeDiv plugin:

using System;
 
class _
{
  static void Main()
  {
    Console.WriteLine("< Happy New Year! >");
  }
}

and some XML:

<?xml version="1.0" ?>
<message>
  <text>Happy New Year!</text>
</message>