How do I evaluate Bible software?

Rubén Gómez at BSR forced me to this! Coming to my own defense, I think I have been transparent about my approach to evaluating Bible software (e.g., here), but I no doubt could stand to be more explicit. So, how do I evaluate Bible software?

I don't have a long list of criteria , and I primarily take a functional approach. My basic question is this: What is the best way for my students and me to accomplish biblical text tasks/research that we do on a regular basis? Of course answering that question is complicated by a host of underlying conditions.

Value: I've been teased for focusing so often on FREE stuff on this blog, but I will admit that value is an important category for me. This is somewhat of a subjective criterion, and it is not simply a matter of accumulating content in a package. It is important, however, for at least a couple reasons:

• Most of my seminary students are already racking up big student loans, and becoming a worker in the church (and this really includes seminary profs too) is not the road to riches. How can they most reasonably obtain the ministry tools they will need?
• Everything has a cost. I'm always trying to temper my perceived needs (for more hard/software) with the real needs in the world around me. How much a year am I spending on tech stuff? How much a year am I donating to relieve hunger and suffering in the world? There is also the cost of time. If the software is so buggy, complicated or slow that it becomes a time sink, then it has less value to me.

Quality content is important for me as well. I have been fully explicit about what I consider minimal resource needs for my students here. Basically I am looking for reliable original language texts (text critical versions is a plus) that are well and reliably tagged. Students should have good lexicons that provide more than simply a gloss. There should be a variety and range of English translations. Additional resources that augment the study of biblical texts include commentaries and topical dictionaries, extra-biblical literature (e.g., Apostolic Fathers, Pseudepigraphal works, Philo, Josephus, etc.), and atlases. The more these are linked and integrated, the better help they are.

Ease of use is critical. I'm sort of a techy/geeky sort (surprise!) and can intuit or deduct my way through most software, but a know a lot of my students have dropped money in Bible software... and then never figured out how to use it. Training resources are good, but clear and obvious ways of getting to the desired results that don't require any instruction are better. Ease of use thus includes things like interface, management of resources, consistency of application, ability to organize and save commonly used resources, speed and responsiveness of application, and customization.

Everything about Bible software can pretty much be grouped under those three headings of value, quality content, and ease of use. The next question, then, is what are the typical tasks that the software should be able to handle. Here are some examples.

1. Sometimes I want to scan a large chunk of text. (Read Mark 16.) Sometimes I want to focus on a single verse. (Compare Mark 16.6 in Greek and a number of versions.) Sometimes just a word. (Analyze the word ἠγέρθη.) Sometimes I want to compare this text with similar passages. (I.e., synoptic parallels) The software should let me make such changes in focus quickly, easily, and consistently.
2. Let's study that word ἠγέρθη. What does the lemma mean? (Here's where a good lexicon is needed.) What is its grammatical form here? What does it mean in the passive as it is here? (What is a "divine passive," and does it apply here?) How do various translations render it? What are possible synonyms? How is it similar to / different from ἀνίστημι? (Can I have a graph comparing the use of these two words?) How is ἐγείρω / ἠγέρθη used elsewhere in Mark?
3. Still in Mark 16.6, what is known about Nazareth? Find it on a map. (On a map in the program or linked out to an online map.) Where else is it mentioned in Mark, the NT, the Bible, the extra-biblical literature?
4. How can I notate and save the work I've done? What are the options for marking up the text? For making my own notes that are attached to the text? For exporting text to a word processor?
5. What other issues should I be aware of relating to Mark 16.6? Are there text critical issues? Do scholars note anything special?

There are other types of exercises (especially dealing with OT in NT stuff and comparison of Hebrew/Greek), but thoroughly answering all those questions would give a good insight into how (easily) the Bible software works, the resources available, and their success at returning quality results. That's how I evaluate Bible software.

UPDATE: 2008.09.18: David Lang has done a nice job on the Accordance blog demonstrating how he uses Accordance for working at some of the typical tasks I describe.