Software to decipher ancient documents

BTW, for those of you who may have this blog in your RSS list and have given up hope on seeing any post from here again... Since my last post on July 23, I have sold a house (cleaned, fixed, packed, moved), bought a house (moved in and still unpacking... got the Internet and wireless up right away!), brought our eldest to start college (more packing, moving, etc.), taught a 2+ week-long intensive Greek class, and am struggling to finish up some reports and a publishing project. I've got a whole bunch of stuff backed up, but here was a quick and easy link just to show I still exist. (>>> I blog, therefore I am?)

Excerpts from a Reuters account (and HERE is the full article):

By Ari Rabinovitch Ari Rabinovitch – Wed Sep 2, 9:51 am ET

BEERSHEBA, Israel (Reuters) – Researchers in Israel say they have developed a computer program that can decipher previously unreadable ancient texts and possibly lead the way to a Google-like search engine for historical documents.

The program uses a pattern recognition algorithm similar to those law enforcement agencies have adopted to identify and compare fingerprints.

But in this case, the program identifies letters, words and even handwriting styles, saving historians and liturgists hours of sitting and studying each manuscript.

By recognizing such patterns, the computer can recreate with high accuracy portions of texts that faded over time or even those written over by later scribes...

It concludes with this interesting observation:

Uri Ehrlich, an expert in ancient prayer texts who works with Bar-Yosef's team of computer scientists, said that with the help of the program, years of research could be done within a matter of minutes.

"When enough texts have been digitized, it will manage to combine fragments of books that have been scattered all over the world," Ehrlich said.

Google Squared and Biblical Studies

Another interesting new project from the Google folks is called Google Squared. What it does is attempt to create a grid of related data based on search parameters entered by the user. According to its developers:

Google Squared is a search tool that helps you quickly build a collection of facts from the Web for any topic you specify.

• Facts about your topic are organized as a table of items and attributes (we call them "Squares" for fun).
• Customize these Squares to see just the items and attributes you're interested in.
• See the websites that served as sources for the information in your Square.
• Save and share Squares with others.

There are some interesting examples of what it can do here, but I thought I would try some examples related to biblical studies. I was trying to think of some things that might be interesting in a grid, and I first tried listing the codices "alexandrinus sinaiticus vaticanus bezae." As you can see in the results below, it got some aspects correct but missed badly on others.
Fortunately, you can easily delete rows or columns and just as easily create a new row or column with your own description.
It also has a nice feature (when signed in to a Google account) of being able to save the grid. For example, if you click HERE, you can see a grid I generated using various English versions of the Bible. The results seem a bit more apt than my previous example, but there are some strange ones here too. (LOLcat is a textual basis?)
As an another experiment, I tried listing each of the four Gospels and came up with THIS. It does not provided particularly satisfying results. So, using another option with this tool, I started out with an empty square and built up my own grid by row and added columns using suggested parameters. (See the graphic below or click HERE.)
Again, some odd results... a picture of Matthew McConaughey for Matthew and Luke Skywalker for Luke? The descriptions, mostly from Wikipedia, are more often correctly linked, but using the suggested column parameters of Date of Birth and Died provided a very strange mix.
So far this tool has provided some interesting connections but mostly it has provided comic relief. So I tried one more query that actually did return some perhaps insightful results. I typed in "Mark 6:34" and THIS is what I got. It appears to have taken the more significant words from that verse and returned separate analysis for each. That might have some value... (Different verses returned quite different results though...)
BOTTOM LINE: You would probably have to do quite a bit of customization to generate useful results, but some of the returns may be of occasional interest.
If you come up with an interesting query with helpful results, please share them here.
[HT: StephensWeb]

On the horizon: Google Wave

Google Wave looks very interesting. It is an email client (replacing Google Mail?), but it is also something of a collaborative document editor. I took a look at part of the developer announcement, and here is what you might want to know for now.
Traditional email works like this:
I.e., one person sends an email to one or more persons much in the way that traditional snail mail has worked. It's just easier to write to many persons at one time and to respond to messages. Google Wave is conceptualizing a different model which looks like this: I.e., it thinks of a conversation as a centrally (i.e., server-based) object which involves two or more persons. The huge benefit of this approach is that it allows a much clearer view of conversations, and anyone can be invited to join the conversation at any point and have the whole conversation in front of them. Conversations, instead of being a collection of "forward/reply to all" emails with a bottom up chain of earlier messages, will now look like threaded message boards. In addition, participants can jump in at any point and respond to particular parts of a message, and it will all be clear in its threaded outline. In addition, this is all happening in real time, so, in effect, it can also replace your instant messaging devices with the advantage of providing a clear conversation among multiple participants. (NOTE: This looks to me to be a much superior method as compared to carrying on conversations in Twitter, AIM, Facebook, etc.)
Google Wave is also intended as a way of sharing and editing text, photos, maps, etc. with others. The API is open, so they are leaving room for further development to extend its capabilities with other gadgets.
Google Wave is supposed to be available later this year, and I am eager to try it out. In particular, from an educational perspective, this is the best way I have seen to carry out threaded discussions and allow for sharing of resources and do it all in real time if desired. At our seminary, we are trying to think about how we might conduct classes online in cases of emergency. (Think of a blizzard closing campus or a H1N1 flu epidemic causing restrictions.) Google Wave would be a great way to conduct discussion for an online class.
UPDATE: 2009.06.01: HERE is a great overview of Google Wave terminology and capabilities.

On the horizon: BING

Bing is, in many ways, Microsoft's response to Google search, and it will replace Microsoft's LiveSearch. However, it is promoting itself not as a "search engine" but as a "decision maker." I.e., it intends to process information to help you make a decision about such things as (and here is where the initial focus appears to be) shopping, health, travel, and local businesses. If you click on the logo above, you can see the brief video promo, and you'll note how it tries to incorporate reviews with the shopping or local search experience or use 'expert' sites for health info or provide 'farecasts' with travel planning. (The travel part looks to be a combination of Expedia and Farecast, both of which are Microsoft related sites.)
I recently mentioned Wolfram|Alpha which bills itself as a "computational knowledge engine," but its value for biblical studies is rather limited, and I'm guessing that bing won't be of any particular extra benefit in this field either unless you find its presentation of results more appealing.
More info here and here.

Wolfram|Alpha and Biblical Studies

There has been considerable buzz lately about WolframAlpha, a "computational knowledge engine." Designed by the brilliant guy who developed Mathematica used in serious mathematical calculations, it is a web search engine that, unlike Google and others, does not direct you to other sites (though it does sometimes offer to send you to Wikipedia), but attempts to calculate and compute your query and return what you actually are hoping to find. It has just gone live today (2009.05.18) on the web, and I thought I would run some biblically related queries through it to see what value it might have for the kind of work I do.

As you can see above, it understands "Jesus" as a person and gives suitable biographical information you might expect for any historical person. Interestingly, it picks up his name in Aramaic but provides modern names for birth and death places. It appropriately identifies some uncertainty in the birth date but is quite confident about the death (though this is a typical date surmised by many).

A query for "Bible" turned up this result. Not much there... but its attention to the sales number hints at what WolframAlpha is more interested in.

I tried searching the "Gospel of Mark," and it did not understand my query. Looking for "Qumran" did no better.

WolframAlpha does like things related to dates. "30 ce" will get you the following.
Also check entities that are associated with dates, e.g., "Roman Empire."

It does seem heavily tilted to USA-centric matters, and it does best at things requiring computational work: numbers, dates, and especially business matters. It has some adeptness with geographical concerns. I tried "jerusalem to jericho." It assumed Jerusalem, Israel and Jericho, New York, but it understood that I may have intended a different Jericho and provide a link to correct it to Ariha, West Bank. Here is the result, and it helpfully provided distance, local times, a map, populations, and elevations.

It also can compute n-grams. According to Wikipedia, "N-gram models are a type of probabilistic model for predicting the next item in a sequence. n-grams are used in various areas of statistical natural language processing and genetic sequence analysis." I was pleased to see that WolframAlpha is unicode-friendly, so it was able to accept unicode Greek. Here is the n-gram of John 1.1 in the Greek. Perhaps linguists will be excited about the potential for this tool, but this word-level bigram can give you an idea of what it might be able to do.

BOTTOM LINE: WolframAlpha is not going to replace Google or other search engines you may be using for doing research related to biblical studies. It does have some interesting potential for computations involving dates or words. Play around with it and let us know if you discover something unique that it can do for the biblical studies field.

Research on the go (or Photocopiers are so 20th century)

Most of you have probably already figured this out, but I find I need to remind my students occasionally about some research tools we now have that were not readily available even five years ago. So, let's say you are in the library or a bookstore and you find something you want to remember. The 20th century way would be to get out paper and writing utensil or, if you were in the library, bring the book to a copy machine and make some paper reproductions. What does the true 21st century researcher do? [Check Tommy Keene's (TK) comment to this post.]
If it's short enough:

• I use my PDA and add it as a note.
• Send a text message to myself on my cell phone.

If it's a bit longer:

• Use my PDA and make a voice recording.
• Send a voice message to myself on my cell phone.
• Send a voice message to your drop.io box. (Thanks, TK! I've been using drop.io for files, but I was unaware that you can also drop voice mails as well. [drop.io is free, but you do have standard phone charges.])
• I had really liked using Jott (I'd blogged about it here) because it did voice to text conversion and sent them as text messages or emails. (I really hate trying to type out text messages on my phone...) Jott is dropping their free, basic plan, however. A standard plan will now cost $3.95/month which is not bad at all, but I wasn't using it quite enough to justify it. Do check out Jott Links, though. Using your phone, you could use the voice to text to set appointments for your Google Calendar, add items to your Remember the Milk list, update Twitter or FaceBook, or name some item/book and come home to find a link to that item on Amazon.
  

If it's a chunk of text or a visual:

• In our seminary library, we now have a copier that saves files as PDFs and delivers them to my network folder. That's sweet...
• No PDF copier? Use your cell phone camera or (if, like me, your cell phone is one of those ancient ones that only is a phone) take a digital camera picture. (Now, how do you get the pic from your phone to your PC? Good advice HERE.)

But what if it's really the text and not just a picture of it you want? Open your picture in optical Character Recognition (OCR) software.

• I have an inexpensive scanner that comes with some basic OCR software. I can open JPG, PNG, TIFF... and it does a pretty good job of getting everything into text. Note that this also works well if you want to convert PDFs which are 'images' and not text. Use a screen capture utility (suggestions here) to take a pic of the PDF, then feed it through the OCR.
• Don't have a scanner or OCR software. Check the Wikipedia OCR entry, and note the list of software (some of which is free/open source) available.
• Don't want to bother with dedicated OCR? Want to try full-featured note-taking software? Then remember that Evernote (there is a free version--my review here and another here) and OneNote ($59 for Microsoft OneNote Home and Student 2007 or $80 as part of Microsoft Office Home and Student 2007) have a built-in tool that can identify and extract text from pics.

For biblical studies sorts, however, there is a problem... None of the OCR software (unless you buy very expensive professional stuff) knows how to deal with Greek or Hebrew text. (I do know of HOCR which is free OCR software for Hebrew, but I have not tried it.)

Do you have any research tools suggestions for the 21st century? Help save some paper and share them here!

BTW, as TK points out in the comment, did you see the Zotero icon up in the address bar of this (and any other) blog post? Click on that folder icon to see the Zotero options for saving info.

More web search alternatives

I have been keeping a running list of search engine possibilities. MyGodSearch is a somewhat newer one that claims to search over 2000 web sites. It's basically a Google custom search that "searches religious and web 2.0 sites, for educational and ministry work." It also has a Delicious Tag Cloud search that has plenty of helpful categories.
Here's my running collection of search engines that I find useful.

Searchme

View in searchme: full | lite

For some other specialized search engines, here is a list of 13 sites for finding such things as music, videos, recipes, quotes, and more.

New online search sites for academics

A couple new search sites have emerged that are intended for more serious, academic work. RefSeek is fully functioning, and in searching for biblically related stuff, it does return results one would hope to see and does omit a lot of junk. ReferenceExtract is still in development.

RefSeek (rĕf-sēk) is a web search engine for students and researchers. RefSeek aims to make academic information easily accessible to everyone. RefSeek searches more than one billion documents, including web pages, books, encyclopedias, journals, and newspapers.

Rference Extract (aka RefEx or REX) is envisioned as a web search engine, like Google, Yahoo and MSN. However, unlike other search engines, Reference Extracts will be built for maximum credibility by relying on the expertise and credibility judgments of librarians from around the globe. Users will enter a search term and get results weighted towards sites most often referred to by librarians at institutions such as the Library of Congress, the University of Washington, the State of Maryland, and over 1,400 libraries worldwide. This grant will support planning for Reference Extract and building the foundation necessary to implement it as a large-scale, general user service.

(HT: Dan Cohen)

Library Management Tools

I'm trying to get myself, my library, and my notes organized, so I've been looking at a number of library management tools. There are a number of dedicated programs like Endnote for bibliographic work, and there are integrated programs like Orbis, Ibidem, and IbidPlus which are wonderful to use within the Nota Bene word processing program. (If you are a regular NB user, I'm guessing this is already a no-brainer choice.) For simply creating bibliographic entries in a variety of formats, there is OttoBib.
To organize your library, I've previously mentioned Amazon's YourMediaLibrary which is a helpful (especially if you buy a lot of stuff from Amazon) but limited organizer. LibraryThing is a much more versatile offering, and I like the way it looks and the social aspects of the site, but it really isn't intended to be a bibliographic tool. While free for 200 book listings, the cost ($10/yr or $25 lifetime) is not prohibitive either. Zach Hubert (of Zhubert > The Resurgence Greek Project fame) has recently released ShelfServant. It is similar to LibraryThing for cataloging, but it doesn't have quite as many extra features. What it does have, however, is an easy way to keep track of books you loan and a one-click export of all your bibliographic data to an Excel spreadsheet file. You could use this to add notes. ShelfServant is free for 100 books with minimal costs to expand your collection.
If you are trying to organize your notes beyond books alone, you are either going to want to use the Nota Bene products I described above, or I think you will want to use Zotero about which I recently blogged. I'm still learning all the things this program can do, but so far I'm more and more impressed at its capabilities and potentials. (I'm playing around with running it using the portable Firefox off a USB stick, so that I can have it wherever I happen to be working.)
Oh, to buy all those books you want, don't forget the "Book Burro" addin for Firefox I previously described!
Bottom line: If you use NotaBene, Orbis and Ibidem are the way to go. Otherwise, I think Zotero is going to be my tool of choice.
If you have other or favorite research management tools, let me know!

Migne's Patrologiae Cursus Completus, Series Graeca

Need to find some Greek text out of Migne? Here are a few good places to check.

• The Religion and Technology site has a complete index of PG. (Just a list of volumes. No texts, though you can buy their DVDs.)
  
• In addition to loads of other online links (including PL) to books related to the Bible, Judaism, and Christianity, Mischa Hooker has compiled links to PG volumes available through Google Books. Note that the Google Books version of PG are PDF image files. (You can get "plain text," but you won't be able to read the Greek.)
• "Roads of Faith - Digital Patrology" is a site that is composed in Greek, but it has a very large collection of the PG volumes organized by periods or alphabetically by author. These are PDF files, but they are actual Unicode text. [UPDATE: 2008.07.06: The Roads of Faith site seems to regularly go offline. Try again later...]
• UPDATE 2008.07.06: As noted in the Kent (from Logos),Logos Bible Software recently announced work on the first twenty volumes of the Patrologia Cursus Completus, Series Graeca. At this time it is on prepub for $400, and as a Libronix doc, it will have the benefit of being cross-linked with other resources. (E.g., with English translations, but I don't think we should expect it to be morphologically tagged!) There is a nice description of the contents of these 20 volumes on the prepub page, and Rick Brannan provides a full table of contents here in PDF.
  

Zotero - New research management tool

In an earlier post, I noted some helpful online research tools. I just discovered Zotero which I think surpasses any other research management tools I have seen. It is a free, Firefox extension, but it has some astounding capabilities. You can take notes (both highlighting and sticky notes), export bibliographic data, and create links with online and local resources. Be sure to take the tour to see some of its present and projected capabilities. You will definitely want to check it out.