What's coming next for Overview

We’ve been doing a lot of user testing recently, and for May and June month we’re focusing on integrating your feedback. Lots of big stuff coming in the next month or so.

Integrated search.

This is high on the list of user requests. Overview will shortly have a full-text search field: type in your search word or phrase, and the system will highlight all the documents in the tree that contain that term, and load those documents into the document list. If you find something interesting, you can turn your search results into a tag with one click.

New document list.

It hasn’t always been obvious that the lower left panel of the main screen is a list of documents, showing all the documents in the currently selected folder or tag. Nor has it been clear how to go through the documents in this list quickly (At the moment, the fastest way is to use the j and k keys.) We’re going to have a much more descriptive list of documents, including title, text snippet, and suggested tags (see below) for document in the list.

Suggested tags.

Currently, each folder and document is described by a list of characteristic words, the words that make that folder or document different from all the others. We’re expanding this concept into a list of suggested tags, with a super fast way to apply them to a document, or all documents in the list.

Tablet support — and a new look!

Any way you slice it, going through a document set involves a lot of reading. We’re optimizing the UI for reading on tablets, with a new full screen document mode, and an interface designed for touch. Plus, we’re completely redesigning the site with a clean look that’s easy on the eyes.

We’re very excited about these upcoming changes, because they will make the system much faster and easier to use. The easiest way to know when they arrive is to follow @overviewproject on Twitter.

VIDEO: Text Analysis in Transparency – a talk at Sunlight Labs

Video of talk at Sunlight Labs at the Sunlight Foundation, Washington DC. (Slides.)

This talk is about how text analysis and natural language processing is being used in journalism, open government, and transparency generally.

The first part of the talk is a survey of existing public projects, and the algorithms behind them, including

  • Churnalism detects plagiarism in the news (or press releases!)
  • Many Bills automatically classifies the sections of bills, to detect pork barrel projects
  • Docket Wrench analyzes the comments on proposed regulations
  • NewsDiffs watches for changes in published articles
  • FEC Standardizer automatically cleans campaign donor names
  • MemeTracker tracks political quotes across the whole web, as they mutate

And of course, there’s a brief demonstration of Overview, and a discussion of the algorithms behind it. (First time here? See how Overview works for investigative journalists)

Finally, there’s a great a discussion of where data-driven transparency is going now — or, what should we work on next? How do we know we are working on the right data sets and the right tools? How can we evaluate the impact of transparency projects? The talk ends with a throwdown — the Transparency Grand Challenge!