An audiovisual collection is hard to describe for a simple reason: you have to watch or listen to it to know what it contains. A sheet from a file is read at a glance; a half-hour newsreel or a recording of a session do not state their content until someone plays them in full. An archive of newsreels, a sound library of interviews, a collection of event video recordings —each one holds inside names, dates and topics spread across minutes of image and sound, not on a cover.
That is why these collections are cataloged slowly, and often go uncataloged. Playing each file in full to describe it does not fit in the team’s time, and meanwhile the material piles up, often already digitized but without a record that makes it findable. That is the case Janium Collect addresses when the collection is audiovisual.
What the description starts from
Collect does not “watch” the video or “hear” the audio the way a person would. It works with two derivatives of the file that a language model can read.
The first is the transcription: the voice in the audio —whether a sound recording or a video’s track— is turned into text with automatic speech recognition, and that text becomes the content from which the metadata is extracted. The second, in the case of video, is the per-scene frames: instead of one frame every so many seconds, the system detects when the image changes significantly and takes a representative frame from each stretch. This way the model sees a sequence of images that sums up what happens on screen, supported by what is said.
The mechanism of each mode is already described in detail in two earlier posts —how the transcription works and the choice between transcribing locally or in the cloud in «From a sound recording to a catalog record», and how the frames are sampled and the two paths combined in «From a video to linked records»—. What matters here is the use case: what an audiovisual file gains from applying both.
One record for the whole, and one for each part
The thing that sets an audiovisual collection apart is that the content is almost never a single thing. A newsreel has several stories. A recorded session has several agenda items. An interview has turns. Describing all that in a single record yields a real but poor record: whoever looks for a specific story, an intervention or a topic in that half hour does not find it, because it was subsumed into a general summary.
Collect approaches this with two levels of record. By default, each file produces
one record: the global summary of the content, which for much material is
enough. When the collection warrants it, segmentation is enabled per
institution: besides the global record, the system splits the material —into
interventions for audio, into scenes for video— and catalogs each segment as a
child record linked to the parent. The link is explicit and bidirectional: each
child references the parent with the field that matches the format
(dcterms:isPartOf in Dublin Core, 773 in MARC), and the parent references its
children without embedding them. Each segment stands as a record that can be
consulted on its own, and at the same time it is known which recording it is part
of.
Here a rule Collect applies throughout the process comes into play: do not fill a field with what it cannot support. A fragment in the middle of a recording rarely carries its own title, so the model leaves those fields empty. On the audio path, where segmentation has already deduced a topic and a summary for each intervention, those gaps are filled deterministically, and the record for the whole can be described by aggregating from its parts: its subject is the union of the topics, its description the sum of the summaries. On the video path, the child of each scene is cataloged from its frames, without that filling pass. In both cases it is description derived from what was already extracted, not a second round of guessing.
In the format the institution already uses
None of this changes the collection’s description standard. The output comes out in the format the institution uses: Dublin Core for a simple description, ISAD-G for an archive where provenance and hierarchy matter, MARC21 for a library catalog. The child record comes out in the same format as the parent, not in a separate one.
Part of the record, moreover, does not depend on the content at all. The file’s technical data —duration, codec, resolution— are read directly from the container, without going through the model, because they are measurable facts; in MARC they feed the fixed fields for audiovisual material deterministically. The resource type —whether it is a sound recording or a moving image— is also set that way, not decided by the model. That leaves the model only the part that requires reading the content, which is where it contributes.
Limits
Automatic transcription has error, and that error grows when the audio is poor: background noise, overlapping voices, a distant microphone produce text with flaws that carry over into the extraction. The description of the visual is done from some frames, not the whole video: they capture the subject of each scene, not its nuances, and a brief shot or a gesture may go unreflected. Scene segmentation is heuristic —it is based on when the image changes, which approximates the thematic cuts but does not always coincide with them—, so it serves as a starting point to navigate the material, not as a definitive editorial segmentation.
When an audio has almost no speech —music, ambience— the system does not force a description out of a poor transcription: it relies on the file’s metadata and flags the case, so as not to fabricate content that is not there. And as in the rest of Collect, what comes out are proposed records that someone reviews. Transcription and per-frame description assist the work; they do not replace the expert viewing of heritage material. A historical audiovisual collection calls for the judgment of someone who knows what they are seeing and hearing —an attribution, a date of the recording, the identification of a person or a place in frame— and that last word remains human. What changes is the starting point: review begins from a structured, linked draft, not from scratch, and above all from material already described enough to find it.
To continue the conversation
If your institution has an archive of newsreels, a sound library or a collection of event video waiting to be described, and the bottleneck is having to watch or listen to each piece to catalog it, this is what this workflow tries to ease, without taking the final decision away from whoever catalogs. Write to us at info@janium.com; we would like to know what your audiovisual material is like, in what format you describe it and how clean the audio is, because that changes what can be expected.