Oddmuse and WebDAV

I have to start keeping notes somewhere. As this is work in progress, I’ll try to apply WikiAsYouLearn. And learn I must, for I know nothing of the WWW Distributed Authoring and Versioning working group (WebDAV). See RFC 2291 for an introduction, RFC 2518 for the real spec.

Source:

Tags:

Motivation

Oddmuse has the raw=1 parameter to facilitate the implementation of user agents that are not generic web browsers. Other wikis offer XmlRpc, and other wikis offer nothing so that you have to resort to screen-scraping (parsing the HTML).

While writing WikiRemote for Emacs, I ran into this problem. Every wiki engine required a different interface. On top of that, I could not use simple file access to create, read, and edit pages. Instead, I had to write a little user-agent in EmacsLisp – HttpGet and HttpPost. That user-agent had to deal with redirection, transfer-encoding, server hanging up, etc. It was tiresome.

WebDAV promises a way out: Accessing wikis should require no special user-agents. Various operating systems or tools allow access to WebDAV hosts as filesystems. Once the page database is accessable via the filesystem, any editor can be used to modify pages.

This does not solve the problem of differing text formatting rules for different wikis, but it does solve two other problems:

  1. The same interface to all wikis.
  2. Using your favorite editor and other tools that act on files to search and edit wiki pages.

Tools

I hear the OS X Finder can mount WebDAV. After some experiments I have decided that the Finder is totally useless in this respect. I also found some more Finder discussion at ars technica.

I noticed that Goliath suffered the same problems. Just hanging there, doing I don’t know what. Probably trying to format my harddisk. :)

GUI clients:

Command-line clients:

Some sniffing:

References

From RFC 3253, Versioning Extensions to WebDAV:

To maximize interoperability and the use of existing protocol functionality, versioning support is designed as extensions to the WebDAV protocol [RFC2518], which itself is an extension to the HTTP protocol [RFC2616]. All method marshalling and postconditions defined by RFC 2518 and RFC 2616 continue to hold, to ensure that versioning unaware clients can interoperate successfully with versioning servers. Although the versioning extensions are designed to be orthogonal to most aspects of the WebDAV and HTTP protocols, a clarification to RFC 2518 is required for effective interoperable versioning.

I will focus on the non-versioning support for the moment.

XML or HTTP Headers

It says in RFC 2518:

In HTTP/1.1, method parameter information was exclusively encoded in HTTP headers. Unlike HTTP/1.1, WebDAV encodes method parameter information either in an Extensible Markup Language (XML) [REC-XML] request entity body, or in an HTTP header. The use of XML to encode method parameters was motivated by the ability to add extra XML elements to existing structures, providing extensibility; and by XML’s ability to encode information in ISO 10646 character sets, providing internationalization support. As a rule of thumb, parameters are encoded in XML entity bodies when they have unbounded length, or when they may be shown to a human user and hence require encoding in an ISO 10646 character set. Otherwise, parameters are encoded within HTTP headers. [...]

In addition to encoding method parameters, XML is used in WebDAV to encode the responses from methods, providing the extensibility and internationalization advantages of XML for method output, as well as input.

Locking

Good news:

A WebDAV compliant server is not required to support locking in any form.

Oddmuse does, however, support the locking of pages and the locking of the entire site. I guess we should support this. This is user authentification, however – not the exclusive and shared locks described in the RFC.

Methods

A quick summary of methods used, and how I plan to deal with them.

Note:

At home, the OPTIONS request is answered by the Apache webserver without my script getting a chance to answer – this is why right now the answer is just plain wrong.

On my test server http://www.emacswiki.org/cgi-bin/test/dav/ the situation is worse: Not only is OPTIONS answered by Apache (and therefore doesn’t reflect what my script can handle), it also disallows all methods except for GET POST OPTIONS PROPFIND – essentially a read-only server.

seems like this has been fixed now – see below

OPTIONS

Do not send DAV header (which I could not get to work anyway) because we don’t implement several methods. Send ALLOWS header with the stuff we do implement.

The problem is that Apache doesn’t let us handle OPTIONS according to a blog I found. [1] And yet, when comparing cadaver and OSX Finder, all I see is that cadaver sends OPTIONS and a PROPFIND, where as OSX Finder sends OPTIONS and gives up – because the default OPTIONS response Apache sends back does not include PROPFIND!

The author filed a bug, it seems. [2] Argh! So if a client chooses to actually pay heed to what OPTIONS is saying, it will never work unless you patch Apache. The bug’s open since 1999 if I read the text correctly. If you are bothered by this, please register at the site and cast three votes for this bug. :)

I can add headers to the response using mod_headers, but I cannot change the Allow header.

It is possible to add a DAV: 1 header to the OPTIONS response. In /etc/httpd/httpd.conf, enable mod_headers. In the .htaccess file of the directory where the Oddmuse script resides, add:

Header set DAV 1

This will allow OSX Finder to continue on to the empty wiki. There are still problems when the wiki is non-empty.

See OS X Finder and WebDAV for more.

it seems like this has been fixed now. I used the following cgi script:

#!/usr/bin/perl

use CGI qw(:standard);


if($ENV{'REQUEST_METHOD'} eq 'OPTIONS') {
      print header( -allow          => "GET OPTIONS",
		    -DAV            => 1,
		    -status         => "200 OK", );
}

and the following shell command:

echo -e "OPTIONS /cgi-bin/webdavtest.cgi HTTP/1.1\nhost: localhost\n" | nc localhost 80

and received:

HTTP/1.1 200 OK
Date: Wed, 25 Feb 2009 02:05:30 GMT
Server: Apache/2.2.9 (Debian)
Dav: 1
Allow: GET OPTIONS
Vary: Accept-Encoding
Content-Length: 0
Content-Type: text/html; charset=ISO-8859-1

BayleShanks

Status: Broken

This example is wrong, however, because the Allow header contains bogus information. It was server by Apache directly, not via the script.

OPTIONS /cgi-bin/wiki/dav HTTP/1.0

HTTP/1.1 200 OK
Date: Sun, 28 Aug 2005 18:20:27 GMT
Server: Apache/1.3.33 (Darwin)
Content-Length: 0
Allow: GET, HEAD, POST, OPTIONS, TRACE
Connection: close

PROPFIND

What properties should a wiki page have? Not all of them are interesting:

ts: Timestamp of last edit

summary: Summary provided for last edit

username: Username of last author

ip: IP number of last author

host: Hostname of last author

languages: Languages used, if identified

revision: Current revision

lastmajor: Current major revision

oldmajor: The major revision before the current one

minor: Whether the last edit was a major edit (1 if revision ≠ lastmajor)

Maybe we should limit ourselves to what RSS and the WikiModule can do.

Status: Done

We read each file to get at the properties, and then we cache the info until any page is changed (by comparing it with the timestamp of the index file). This is slow on active wikis.

We also provide an Etag header with a md5 hash of the request and handle If-None-Match headers appropriately. RFC 2518 is explicit in this respect, however:

The results of this method SHOULD NOT be cached.

Oh well.

PROPFIND /cgi-bin/wiki/dav/Schr%c3%b6der HTTP/1.1
Host: localhost
Depth: 1
Content-Type: text/xml; charset="utf-8"
Content-Length: 97

<?xml version="1.0" encoding="utf-8" ?>
<D:propfind xmlns:D="DAV:">
  <D:allprop/>
</D:propfind>

HTTP/1.1 207 Multi-Status
Date: Tue, 30 Aug 2005 19:33:47 GMT
Server: Apache/1.3.33 (Darwin)
Etag: 22ReYtqqT7pS59dYQjv5QQ
Transfer-Encoding: chunked
Content-Type: text/html; charset=UTF-8

291
<?xml version="1.0" encoding="utf-8"?>
<D:multistatus xmlns:D="DAV:">
  <D:response>
    <D:href>http://localhost/cgi-bin/wiki/dav/Schr%c3%b6der</D:href>
    <D:propstat>
      <D:prop>
        <D:creationdate>1970-01-01T00:00:00</D:creationdate>
        <D:getcontentlength>9</D:getcontentlength>
        <D:getcontenttype>text/plain</D:getcontenttype>
        <D:getlastmodified>Mon, 04 Apr 2005 23:06:48 GMT</D:getlastmodified>
        <D:resourcetype/>
        <D:displayname>Schröder</D:displayname>
        <D:getetag>1112656008</D:getetag>
      </D:prop>
      <D:status>HTTP/1.1 200 OK</D:status>
    </D:propstat>
  </D:response>
</D:multistatus>

0

PROPPATCH

Since page properties cannot be edited without editing the page itself, the PROPPATCH method could be handled using a 403 Forbidden status. Then again, using 501 Not Implemented is the generic reponse for all methods that have not been implemented, so that’s what I’ll use.

Status: Done

PROPPATCH /cgi-bin/wiki/dav/mu HTTP/1.1
Host: localhost

bla bla

HTTP/1.1 501 Not Implemented
Date: Sun, 28 Aug 2005 18:02:08 GMT
Server: Apache/1.3.33 (Darwin)
Connection: close
Transfer-Encoding: chunked
Content-Type: text/html; charset=UTF-8

MKCOL

Since Oddmuse does not support subpages, all MKCOL requests will be handled using a 501 Not Implemented status.

A 403 Forbidden would be a possible answer as well, but it gives the impression that it would be possible, in theory, if only you had the correct permissions.

Possibly Oddmuse namespaces might include MKCOL. But not now.

Status: Done

MKCOL /cgi-bin/wiki/dav/mu HTTP/1.1
Host: localhost

HTTP/1.1 501 Not Implemented
Date: Sun, 28 Aug 2005 18:12:26 GMT
Server: Apache/1.3.33 (Darwin)
Connection: close
Transfer-Encoding: chunked
Content-Type: text/html; charset=UTF-8

GET

Unmodified. Serve raw text.

Status: Done

GET /cgi-bin/wiki/dav/HomePage HTTP/1.0

HTTP/1.1 200 OK
Date: Thu, 11 Aug 2005 02:18:49 GMT
Server: Apache/1.3.33 (Darwin)
Cache-control: max-age=10
Etag: 1123637042
Connection: close
Content-Type: text/plain; charset=UTF-8

muuu

**one**: two

**three**: four

yadda
yadda

Caching:

GET /cgi-bin/wiki/dav/HomePage HTTP/1.1
Host: localhost
if-none-match: 1123637042

HTTP/1.1 304 NOT MODIFIED
Date: Thu, 11 Aug 2005 02:23:07 GMT
Server: Apache/1.3.33 (Darwin)
Content-Type: text/html; charset=UTF-8

HEAD

Nothing new.

Status: Done

HEAD /cgi-bin/wiki/dav/HomePage HTTP/1.1
Host: localhost

HTTP/1.1 200 OK
Date: Thu, 11 Aug 2005 02:24:08 GMT
Server: Apache/1.3.33 (Darwin)
Cache-control: max-age=10
Etag: 1123637042
Content-Type: text/plain; charset=UTF-8

Caching:

HEAD /cgi-bin/wiki/dav/HomePage HTTP/1.1
Host: localhost
if-none-match: 1123637042

HTTP/1.1 304 NOT MODIFIED
Date: Thu, 11 Aug 2005 02:24:36 GMT
Server: Apache/1.3.33 (Darwin)
Content-Type: text/html; charset=UTF-8

DELETE

This should work for pages and replace their content with DeletedPage. The response would have the status 204 No Content. The problem is that a new listing of the pages would continue to list these pages. Therefore we reply using 501 Not Implemented instead.

When applied to the entire wiki, it would to all pages only. The wiki itself cannot be deleted. Since you would have to deinstall the wiki anyway, deleting the entire wiki via DELETE is not supported. It will be handled using a 403 Forbidden status.

Status: Done

DELETE /cgi-bin/wiki/dav/mu HTTP/1.1
Host: localhost

bla bla

HTTP/1.1 501 Not Implemented
Date: Sun, 28 Aug 2005 18:03:19 GMT
Server: Apache/1.3.33 (Darwin)
Connection: close
Transfer-Encoding: chunked
Content-Type: text/html; charset=UTF-8

PUT

I wonder how one would specify properties such as username or edit summary. Does the client use a PUT, followed by a PROPPATCH? In that case I’ll have to revise what I said above.

When applied to the entire wiki, it applies to all pages. The wiki itself cannot be deleted. It will be handled using a 403 Forbidden status.

Another problem: When putting images or the like, Oddmuse would not recognize it. It just saved the binary data. It seems that the application I was using at the time does not send a Content-Type header along. What I did was rely on the filename extension for common images. There’s better code available as part of the Oddmuse:Static Copy Extension; right now it works for PNG and JPEG files, which seems to be the most important ones.

Status: Done (not metadata)

PUT /cgi-bin/wiki/dav/mu HTTP/1.1
Host: localhost
Content-Type: text/xml; charset="utf-8"
Content-Length: 53

this is a test
and and this is
the text of the
test.

HTTP/1.1 201 Created
Date: Sun, 28 Aug 2005 00:33:50 GMT
Server: Apache/1.3.33 (Darwin)
Transfer-Encoding: chunked
Content-Type: text/html; charset=ISO-8859-1

LOCK

I think 412 Precondition Failed indicates that there could be a way of making the precondition not fail. A 501 Not Implemented seems cleaner.

Status: Done

LOCK /cgi-bin/wiki/dav/HomePage HTTP/1.0

HTTP/1.1 501 Not Implemented
Date: Sun, 28 Aug 2005 18:18:21 GMT
Server: Apache/1.3.33 (Darwin)
Connection: close
Transfer-Encoding: chunked
Content-Type: text/html; charset=UTF-8

UNLOCK

Since we don’t support locking, we should indicate failure to unlock. This is not specified in the RFC, however. We could just use 405 Method Not Allowed as defined in RFC 2616:

The method specified in the Request-Line is not allowed for the resource identified by the Request-URI. The response MUST include an Allow header containing a list of valid methods for the requested resource.

The problem is that we do understand the UNLOCK method – we just don’t allow unlocking of resources. Then again, our resources are always unlocked. Therefore we’ll just do what the example in the RFC does: Return a 204 No Content.

Status: Done

UNLOCK /cgi-bin/wiki/dav/HomePage HTTP/1.0

HTTP/1.1 204 No Content
Date: Wed, 24 Aug 2005 17:46:11 GMT
Server: Apache/1.3.33 (Darwin)
Connection: close
Content-Type: text/html; charset=UTF-8

COPY

We will only support copying a page within the same wiki, because we don’t know how to post to a remote wiki. Therefore, the client will have to do this by itself if required.

The wiki itself cannot be copied using this method. It will be handled using a 403 Forbidden status.

We will handle the Overwrite header (T/F). Note that we will treat pages marked for deletion as existing pages!

201 (Created) – The source resource was successfully copied. The copy operation resulted in the creation of a new resource.

204 (No Content) – The source resource was successfully copied to a pre-existing destination resource.

403 (Forbidden) – The source and destination URIs are the same.

409 (Conflict) – A resource cannot be created at the destination until one or more intermediate collections have been created.

412 (Precondition Failed) – The server was unable to maintain the liveness of the properties listed in the propertybehavior XML element or the Overwrite header is “F” and the state of the destination resource is non-null.

423 (Locked) – The destination resource was locked.

We don’t do the locks described by the RFC.

502 (Bad Gateway) – This may occur when the destination is on another server and the destination server refuses to accept the resource.

We could just assume that the destination is an Oddmuse wiki and do the right thing... Maybe later.

507 (Insufficient Storage) – The destination resource does not have sufficient space to record the state of the resource after the execution of this method.

We don’t handle insufficient storage issues.

MOVE

The wiki itself cannot be moved using this method. It will be handled using a 403 Forbidden status.

Single pages can be moved. Maybe only allow this if the Oddmuse Admin Power Extension is available. We need an extensible design for this WebDAV implementation.

201 (Created) – The source resource was successfully moved, and a new resource was created at the destination.

204 (No Content) – The source resource was successfully moved to a pre-existing destination resource.

403 (Forbidden) – The source and destination URIs are the same.

409 (Conflict) – A resource cannot be created at the destination until one or more intermediate collections have been created.

Since we don’t have directories, this is never a problem. We don’t support namespaces, yet.

412 (Precondition Failed) – The server was unable to maintain the liveness of the properties listed in the propertybehavior XML element or the Overwrite header is “F” and the state of the destination resource is non-null.

423 (Locked) – The source or the destination resource was locked.

We don’t do the locks described by the RFC.

502 (Bad Gateway) – This may occur when the destination is on another server and the destination server refuses to accept the resource.

Comments


Please make sure you contribute only your own work, or work licensed under the GNU Free Documentation License. See Info for text formatting rules. You can edit the comment page if you need to fix typos. You can subscribe to new comments by email without leaving a comment.

To save this page you must answer this question:

Please say HELLO.