We’ve been looking at government publications systems lately, partly at the instigation of Beth Noveck, who leads the U.S. government’s Open Government Initiative [1]. Two of the most interesting are the Federal Register 2.0 and New York Law School’s CityAdmin system — they come at the problem from opposite directions, and comparing them is instructive.
Federal Register
The new FederalRegister.gov site, which launched in late July, is a complete redesign based on a Sunlight Labs app contest winner (see here for the full story). It offers very intuitive, consistent navigation that helps users narrow in on and subscribe to a particular information source. It integrates with Regulations.gov and the Unified Agenda, so users can easily go from reading a notice to submitting comments directly into the relevant docket (a very important feature for public participation), and it enables users to view the history of rulemaking activity topically or chronologically. It also integrates with social media sites.
From a usability and integration perspective, it is a smashing win. Here’s a quick tour, just to give you an sense of how it flows. The front page is like an online newspaper, with topic sections and highlighted articles (based on a combination of popularity and editors’ choice). Along the top are links to recent notices, proposed changes to rules and regulations, etc:
We’ll click on Proposed Rules. The resultant page shows some recent proposals — it happens that ones from the Federal Aviation Administration are most recent, hence at the top of my screenshot. Each proposal has a title followed by a short summary:
Clicking on the title brings us to the full text of the proposal:
Fantasize for a moment that you’re in the airline industry. You might want to see what else the FAA is publishing. Clicking on the agency link brings us to the agency’s Federal Register page, from which we can subscribe to an agency-specific data feed…
…or browse recent publications from that agency:
Unfortunately, there’s a catch: maintaining this kind of consistency requires a lot of input standardization and top-down cooperation from the agencies supplying the data. So although the Federal Register code is open source and available, for most jurisdictions it wouldn’t simply be a matter of dropping it in and turning the key. To fit into this system, the data sources need to be disciplined about how they supply their information. (Also, to be fair, there would have to be some abstraction on the code side to handle the different organizational structure and data of, say, a municipal government as opposed to the federal government.)
CityAdmin
At the other extreme is the CityAdmin system. This is a city document repository that’s not even run by the city — it’s run by New York Law School’s Center for New York City Law, directed by Prof. Ross Sandler.
Sandler and his colleagues persuaded most of the major agencies in New York City government to cooperate by essentially offering to do their publication for them, without making many demands about data formats or automation. It’s a very hands-on process: agencies send CityAdmin their latest files (usually copies of whatever word processor format the agency uses internally) at irregular intervals, sometimes with help from CityAdmin staff. The files are simply stored in a repository connected to a Google Search Appliance, with a custom-written web interface wrapped around it. The result is ad hoc but effective:
You check the boxes corresponding to the agencies you want to search in and enter your search terms (logical operators such as AND and OR can be used):
The results come back looking more or less like typical Google search engine results, except that the link might be to (say) a raw WordPerfect file stored in the CityAdmin archive:
The brief summaries shown in the result list are based on the context around the search terms as they appear in the document, rather than being a summary of the overall document. CityAdmin would love to provide a per-document summary instead, but that’s the problem with having non-standard input formats: it’s hard for their software to find a summary or abstract in the document, even if there is one. CityAdmin is working with some of the agencies to standardize on certain conventions, so that the system can provide better summaries at least some of the time. Merely parsing the source format is part of the issue: although the Google Search Appliance is able to parse the WordPerfect files in order to search across them, for example, CityAdmin’s own software doesn’t (yet) know how to read WordPerfect. Thus some of the documents are opaque to the part of the system that most needs to do semantic analysis on them. (The Federal Register code doesn’t have this problem because everything arrives in a known format already.)
Data intake happens irregularly: some agencies send decisions whenever they are produced, others send once per month or whenever their commission meets and issues decisions. Most agencies upload directly to a designated area, but some still send email to a CityAdmin staffer who then does the upload for them. The web site doesn’t offer standard subscription feeds, but there is a paid CityAdmin Express notification service available. It sends you a customized email alert when decisions are uploaded to CityAdmin in agencies of your choice. (This highlights another aspect of CityAdmin: it’s not funded by the City, so it has to make do with private donations, paid subscribers, and a committed staff who — I’m guessing here — probably puts in more hours than they report. As a sign of what determined public access crusaders can accomplish, it’s an amazing example; as a model for how to fund public access to public data in general, though, I’m not sure they’d necessarily want to be emulated!)
Despite its unofficial status and the heterogeneity of its data, CityAdmin has become a valuable resource for public access to New York City agencies. It’s linked to directly from agency pages under nyc.gov, a testament to its acceptance as a de facto publication system, and it receives around 10,000-12,000 visits a month (from 3000-3500 unique addresses). It works because they’ve made CityAdmin very easy for agencies to deliver to, and, where necessary, applied the personal touch: if they don’t hear anything from a certain agency in a given month, one of the CityAdmin staff will call up the right person at that agency — someone with whom they’re probably on a first-name basis by this point — and ask what’s up.
Next steps
Each system suits its circumstances well. While ultimately it would be wonderful if every jurisdiction could deploy a version of the Federal Register software, in practice many are not ready to impose that kind of cross-agency standardization, and the (relatively) cheap-and-quick method used by CityAdmin could provide a lot of access a lot sooner. Our goal at Civic Commons is to help other jurisdictions figure out which way, if either, would be most appropriate for them, and where necessary to help coordinate improvements to the systems to make them more widely deployable. One thing we’re working on — and that we’ll be writing more about here — is integration of user feedback.
The Federal Register uses TenderApp to collect feedback and to publish responses to users’ questions. Their front page prominently invites comments, and one can see from looking in their TenderApp FAQ area that they’ve received and responded to users’ questions. (Curiously, the discussions area is empty, but that may be because they clear out the discussions as soon as they are turned into FAQ responses; I’m not sure and will find out.) CityAdmin currently uses just an email address for feedback, but they’d like to integrate web-based feedback mechanisms into their site — see this conversation on the Civic Commons discussion list. TenderApp was one of several solutions suggested there; we’ll be investigating them with CityAdmin and posting about the results, as getting good feedback systems in place is just about the most important thing any service-oriented site (government or otherwise) can do. Watch this space for more.
–Karl Fogel
