[Discussion] Indexing

[Timotheus]

Cross-referencing will come soon, indexing will undoubtedly follow. So perhaps it is not a bad idea to start thinking about what indexing could and should be in Mellel.

Let me start with a request and with a suggestion.

The request: I hope that a future version of Mellel will be able to generate any desired number of indexes, ilke the present version is able to generate any desired number of footnote streams.

The suggestion: for those interested in the ins and outs of indexing, a storehouse of information is the site of the American Society of Indexers: http://www.asindexing.org/site/index.html .

[donb]

I very strongly support the idea of having various modes/types of indexing in Mellel. In particular, I think Maria's suggestions for the three types of indexing would be of very great value -- certainly in my work they would!

Don Broadribb

[Ori Redler]

With Index, the main issues, it seems to me, are first entry or collection of index items; and then the way the index itself is organised.

For the "marking" stage, it seems a natural choice to allow for entry of an item as a main entry, sub entry of something else (which can be selected from a list of existing entries) and as a "see also" (which can be an additional option to the two above).

Regarding index "format" the options should be of two types:
A) style and format (e.g., order of items in a specific entry or entry level (main, sub, see also, etc.). One should also be able to set other options here like sorting by chapter, language, sort group (e.g., when you want groups of index entry A-E, F-I, etc.).

[laup]

Indexing seems to me a perfect example of a nice-to-have. I truly would like to have it; but I doubt that I would use it very often and I could surely live without it, as I have for a long time. I have only indexed one book.

As to what the functionality should be if we had it, it seems to me very important that marking NOT build in a rigid sense of hierarchy. A given word might be used in different ways in the same document, might be seen by some as primary and by others as a special case of something more general. No unique ontology exists and attempts to create one impose a particular perspective.

This suggests to me that any marking of a word as subordinate to another word should be tentative rather than architectual--more of a link than a firm placement in structure. This would require that the tool used allow promotion, demotion, multiple parents, and so on.

[Reiner]

indexing should not only be able with references to page-numbers, line-numbers, margin-numbers, column-numbers but also to auto-titles imho.

[eleuteruiz]

With Index, the main issues, it seems to me, are first entry or collection of index items...

I would suggest to add this as an option in find/replace. People who (like me in Humanities) use Smallcaps for author names would be able to easily mark all of them for index with find/replace. Does it make sense?

[aechallu]

Indexing is nice in theory, but in practice it wouldn't make any difference to me or other people who turn in manuscripts to a publisher. The publisher gives you back a hard-copy prints to proofread and create the index from.

I strongly prefer Mellel to concentrate on:
- facilitating exchange with other scholars (strong RTF style support; hyperlinked, editable PDF output a-la-papyrus; and other export options such as Latex),
- streamlining the menus and improving the GUI,
- extending outline tools,
- better integration in "scholar" suites (devonthink, and an API/scripting to facilitate integration with other bibliographic tools)

I know this contradicts with other's primary use of Mellel as a creator of whole books, but I wanted to state my position anyway...

[Ori Redler]

As to what the functionality should be if we had it, it seems to me very important that marking NOT build in a rigid sense of hierarchy.

Any hierarchy in the index will be purely "local". That is, you would be able to index a word as a sub of another, as an independent entry with its own subs, an independent entry without subs, an entry for different indices, etc. In short, A word indexed would not become an object that is associated with a certain position in an index hierarchy.

[Ori Redler]

With Index, the main issues, it seems to me, are first entry or collection of index items...

I would suggest to add this as an option in find/replace. People who (like me in Humanities) use Smallcaps for author names would be able to easily mark all of them for index with find/replace. Does it make sense?

That would almost certainly be an "Also" option. Not the only way to index, but an additional way to be sure.

[Hans-Reinhard Koch]

Hi there,
I am writing a book and indexing will soon be needed. So I am waiting impatiently. Let me elaborate how I think indexing could work for me.
Several things have to be done.

1. Define an index:
in my book I would like to use a number of indexes: e.g. Index of names, index of places, index of terms, ....

2. Define character and paragraph styles for each of these indexes, to be done in the „Style set > Edit style sets ... „ menu

3. Define index style: Here one can choose options like
layout „uninterrupted“, „interrupted by X lines“ (when first letter of index terms change) or „interrupted by X lines and Capital letter (left/right/center)“ as well as
„Place page numbers after text“ or „at end of line“ and separate page numbers by – , ; _ or whatever
How to handle page ranges like „4, 5, 6, 12, 15, 16, 21, 22, 23, 24, 25“ > leave them or convert to e. g. „4ff, 12, 15f, 21-25“

4. Mark occurrences for indexing: This is more tricky if one wants it to be rather hassle-free. I see the following problem: In the text the name of the same person may occur in different forms, e. g. „Louis Armstrong“ may occur as „Louis“, as „Armstrong“, or just „Satchmo“. In the index I want it always to appear as „Armstrong, Louis“ with page numbers.
The following procedure would work:
I select text, and choose a command: „Edit > Mark for index“ or appropriate short-cut. A dialog window opens with a pop-up menu, from which I can select the appropriate index (default selection being the last used index), and a editable text field which contains the selected text. If this is the text I want in the index, I press „OK“, if not I can edit the text in the field. The moment I start editing an auto-complete mechanism would be nice, offering terms from the index that already exist. I could imagine something like the way Apple mail handles autofilling of mail addresses from the AdressBook. If one has e. g. typed three or four letters, a popup offers all existing index terms starting with these letters. After closing the box the selected text in the manuscript should be highlighted with a color (different from the bibliography or auto-number highlight colors and either preset by Mellel or – better – separately choosable for each index when it is defined)
Sometimes, I will want one text item in the manuscript to appear at different places in the index. E. g. I would want the term „cataract surgery“ to appear in the index both under C „cataract, surgery“ and under S „surgery, cataract“.
To achieve this, i would either select parts of the word and mark each part the word for a different index term. Or I could envisage the above dialog box with more then one text field for entering multiple index terms (doesn‘t look nice to me).
I am sure there are much more elaborate ways to handle hierarchical index lists. However a speedy delivery of something that works without hassle is preferable - and better then not having an indexing feature at all.

5. Create index:
Choose menu item: „Insert > Index > name of defined index“ will create an index with preselected format at the insertion point.
This means there would be an added „Index“ item in the insert menu with a dynamic submenu with the names of all indexes created; a line and a last item „Edit indexes“, where indexes can be created, their names edited and index styles can be edited – somewhat similar to „Insert > auto-title > edit title flows ...“

Real-time up-dating of an index will be very processor time-consuming. It would be sufficient to delete an old index and create the new index at the cursor location again.

[Timotheus]

Let me add three small but rather important things to what has been said already:

- Indexing must be able to perform case sensitive searching: in other words, it must be able to distinguish between the noun "heart" and the name "Heart".
- Indexing must be able to recognize "d'Elba" as a recurrence of "Elba": most word processors with indexing can't.
- Given the fact that especially in old texts words and proper names don't always have exactly the same spelling, indexing must also be able to search for very similar but not identical words / forms.

[shades]

I have followed the discussion and appreciate all the knowledge of participants.

One perhaps simple or even simplistic question I have is about how the indexing fits into the professional indexing situation. When I looked into this 2-3 years ago, professional indexing was still a manual activity. Even the best dedicated software could not replace what the professional indexer did. Has that changed? Is the type of index that Mellel will be implementing to be a bridge between the two worlds? Or is this kind of indexing sufficient for academic production, and is that the goal?

From an old codger who is curious - and delighted to be in the presence of so many knowledgable people (seriously!!).

[zoul]

One perhaps simple or even simplistic question I have is about how the indexing fits into the professional indexing situation. When I looked into this 2-3 years ago, professional indexing was still a manual activity. Even the best dedicated software could not replace what the professional indexer did. Has that changed?

In my opinion it has not. I’m not too much into indexing, but when I last indexed a document I figured out the index should not contain all references to a certain word, but only those that the reader would be interested in (examples are easy to find, just think of this when you read some of your documents next time). Good index requires good understanding of the text, which in turn will always require humans (I hope :).

The specification here is not to “create index”, but “make creating index as easy as possible”. Which usually means make adding words into the index easy, make it possible to create multiple indexes (= multiple views of the content), make it possible to index words with the same meaning together and such stuff.

[This is just a view, I am not much into indexing as I already said.]

[Mats]

Dear all,

I think one should not forget that indexing is a profession and requires not only alot of understanding of the text and topic, but also years of training.

Not referring to the gentlemen and -women in this forum, but when most non-indexers talk about an index, they actually talk about a concordance - which is a completely different animal. The former can not be automated, while the latter can (to a very high degree).

Professional indexes are usually created using dedicated software. The three dominators in this field are the following:

- Cindex: http://www.indexres.com/
- Sky Index: http://www.sky-software.com/
- Macrex: http://www.macrex.com/

If you decide to add an indexing feature to Mellel I would strongly suggest that you take a close look at the features in these programs.

Best wishes,
Mats Broberg

[shades]

Thanks, Mats. You have confirmed my concerns/thoughts/curiosity. And the reason I even raised it, is that someone may be surprised to find that instead of an indexing capability, Mellel provides concordance capability (may still be important, but not the same).