julius/wchmm.c File Reference

Build tree lexicon. More...

#include <julius.h>

Include dependency graph for wchmm.c:

Go to the source code of this file.

Defines

#define COUNT_STEP   500
 Word count step for debug progress output.

Functions

WCHMM_INFO * wchmm_new ()
static void wchmm_init (WCHMM_INFO *wchmm)
static void wchmm_extend (WCHMM_INFO *wchmm)
void wchmm_free (WCHMM_INFO *w)
static int compare_wseq (WORD_ID *widx1, WORD_ID *widx2)
static void wchmm_sort_idx_by_wseq (WORD_INFO *winfo, WORD_ID *windex, WORD_ID bgn, WORD_ID len)
static int compare_category (WORD_ID *widx1, WORD_ID *widx2)
static void wchmm_sort_idx_by_category (WORD_INFO *winfo, WORD_ID *windex, WORD_ID len)
static int wchmm_check_match (WORD_INFO *winfo, int i, int j)
static void add_wacc (WCHMM_INFO *wchmm, int node, LOGPROB a, int arc)
static void wchmm_link_hmm (WCHMM_INFO *wchmm, int from_node, int to_node, HTK_HMM_Trans *tinfo)
static void wchmm_link_subword (WCHMM_INFO *wchmm, int from_word, int from_seq, int to_word, int to_seq)
static void wchmm_duplicate_state (WCHMM_INFO *wchmm, int node, int word)
static void wchmm_duplicate_leafnode (WCHMM_INFO *wchmm)
static void wchmm_add_word (WCHMM_INFO *wchmm, int word, int matchlen, int matchword)
static void wchmm_index_ststart (WCHMM_INFO *wchmm)
static void wchmm_calc_wordend_arc (WCHMM_INFO *wchmm)
void build_wchmm (WCHMM_INFO *wchmm)
void build_wchmm2 (WCHMM_INFO *wchmm)
void print_wchmm_info (WCHMM_INFO *wchmm)

Variables

static int dupcount = 0
 Number of duplicated nodes (for debug only) If defined, do wchmm size estimation (for debug only).
static WORD_INFO * local_winfo
 Temporary work area for sort callbacks.
static int separated_word_count
 Number of words actually separated (linearlized) from the tree.

Detailed Description

Build tree lexicon.

Author:
Akinobu Lee
Date:
Mon Sep 19 23:39:15 2005
Functions to build a tree lexicon (or called word-conjunction HMM here) from word dictionary, HMM and language models are defined here. The constructed tree lexicon will be used for the recognition of the 1st pass. The lexicon is composed per HMM state unit, and various informations about output probabilities, arcs, language model constraints, and others are assembled in the lexicon.

Note that the word "wchmm" in the source code is a synonim of "tree lexicon".

Revision
1.4

Definition in file wchmm.c.

Function Documentation

WCHMM_INFO* wchmm_new (  ) 

Allocate a new tree lexicon structure.

Returns:
pointer to the newly allocated tree lexicon structure.

Definition at line 70 of file wchmm.c.

Referenced by final_fusion(), and multigram_setup().

static void wchmm_init ( WCHMM_INFO *  wchmm  )  [static]

Initialize content of a lexicon tree.

Parameters:
wchmm [out] pointer to the lexicon tree structure

Definition at line 103 of file wchmm.c.

Referenced by build_wchmm(), and build_wchmm2().

static void wchmm_extend ( WCHMM_INFO *  wchmm  )  [static]

Expand state-related area in a tree lexicon by MAXWCNSTEP.

Parameters:
wchmm [i/o] tree lexicon

Definition at line 156 of file wchmm.c.

Referenced by wchmm_add_word(), and wchmm_duplicate_state().

void wchmm_free ( WCHMM_INFO *  w  ) 

Free all data in a tree lexicon.

Parameters:
w [in] tree lexicon

Definition at line 206 of file wchmm.c.

Referenced by multigram_setup().

static int compare_wseq ( WORD_ID *  widx1,
WORD_ID *  widx2 
) [static]

qsort function to sort words by their phoneme sequence.

Parameters:
widx1 [in] pointer to word id #1
widx2 [in] pointer to wrod id #2
Returns:
1 if word[widx2] is part of word[widx1], -1 if word[widx1] is part of word[widx2], or 0 if the two words are equal.

Definition at line 292 of file wchmm.c.

Referenced by wchmm_sort_idx_by_wseq().

static void wchmm_sort_idx_by_wseq ( WORD_INFO *  winfo,
WORD_ID *  windex,
WORD_ID  bgn,
WORD_ID  len 
) [static]

Sort word IDs in windex[bgn..bgn+len-1] by their phoneme sequence order.

Parameters:
winfo [in] word lexicon
windex [i/o] index sequence of word IDs, (will be sorted in this function)
bgn [in] start point to sort in windex
len [in] length of indexes to be sorted from bgn

Definition at line 341 of file wchmm.c.

Referenced by build_wchmm2().

static int compare_category ( WORD_ID *  widx1,
WORD_ID *  widx2 
) [static]

qsort function to sort words by their category ID.

Parameters:
widx1 [in] pointer to element #1
widx2 [in] pointer to element #2
Returns:

Definition at line 367 of file wchmm.c.

Referenced by wchmm_sort_idx_by_category().

static void wchmm_sort_idx_by_category ( WORD_INFO *  winfo,
WORD_ID *  windex,
WORD_ID  len 
) [static]

Sort word IDs in windex[0..len-1] by their category ID.

Parameters:
winfo [in] tree lexicon
windex [i/o] index sequence of word IDs, (will be sorted in this function)
len [in] number of elements in windex

Definition at line 392 of file wchmm.c.

Referenced by build_wchmm2().

static int wchmm_check_match ( WORD_INFO *  winfo,
int  i,
int  j 
) [static]

Compare two words from word head per phoneme to see how many phones can be shared among the two.

Parameters:
winfo [in] word dictionary
i [in] a word
j [in] another word
Returns:
the number of phonemes to be shared from the head of the words.

Definition at line 430 of file wchmm.c.

static void add_wacc ( WCHMM_INFO *  wchmm,
int  node,
LOGPROB  a,
int  arc 
) [static]

Add a transition arc between two nodes on the tree lexicon

Parameters:
wchmm [i/o] tree lexicon
node [in] node number of source node
a [in] transition probability in log scale
arc [in] node number of destination node

Definition at line 463 of file wchmm.c.

Referenced by wchmm_duplicate_state(), and wchmm_link_hmm().

static void wchmm_link_hmm ( WCHMM_INFO *  wchmm,
int  from_node,
int  to_node,
HTK_HMM_Trans *  tinfo 
) [static]

Add a transition from end node of a phone to start node of another phone.

Parameters:
wchmm [i/o] tree lexicon
from_node [in] end node of a phone
to_node [in] start node of a phone
tinfo [in] transition prob. matrix of the from_node phone.

Definition at line 592 of file wchmm.c.

Referenced by wchmm_link_subword().

static void wchmm_link_subword ( WCHMM_INFO *  wchmm,
int  from_word,
int  from_seq,
int  to_word,
int  to_seq 
) [static]

Connect two phonemes in tree lexicon.

Parameters:
wchmm [i/o] tree lexicon
from_word [in] source word ID
from_seq [in] index of source phoneme in from_word from which the other will be connected
to_word [in] destination word ID
to_seq [in] index of destination phoneme in to_word to which the other will connect

Definition at line 641 of file wchmm.c.

static void wchmm_duplicate_state ( WCHMM_INFO *  wchmm,
int  node,
int  word 
) [static]

Isolation of word-end nodes for homophones: duplicate the word-end state, link as the same as original, and make it the new word-end node of the given new word.

Parameters:
wchmm [i/o] tree lexicon
node [in] the word end node of the already existing homophone
word [in] word ID to be added to the tree

Definition at line 696 of file wchmm.c.

Referenced by wchmm_duplicate_leafnode().

static void wchmm_duplicate_leafnode ( WCHMM_INFO *  wchmm  )  [static]

Scan the whole lexicon tree to find already registered homophones, and make word-end nodes of the found homophones isolated from others.

Parameters:
wchmm [i/o] tree lexicon

Definition at line 810 of file wchmm.c.

static void wchmm_add_word ( WCHMM_INFO *  wchmm,
int  word,
int  matchlen,
int  matchword 
) [static]

Add a new word to the lexicon tree. The longest matched word in the current lexicon tree and the length of the matched phoneme from the word head should be specified to tell where to insert the new word to the tree.

Parameters:
wchmm [i/o] tree lexicon
word [in] word id to be added to the lexicon
matchlen [in] phoneme match length between word and matchword.
matchword [in] the longest matched word with word in the current lexicon tree

Definition at line 891 of file wchmm.c.

Referenced by build_wchmm().

static void wchmm_index_ststart ( WCHMM_INFO *  wchmm  )  [static]

Inspect the whole lexicon tree to generate list of word head states for inter-word transition computation.

Parameters:
wchmm [i/o] tree lexicon

Definition at line 1329 of file wchmm.c.

static void wchmm_calc_wordend_arc ( WCHMM_INFO *  wchmm  )  [static]

Scan the lexicon tree to make list of emission probability from the word end state.

Parameters:
wchmm [i/o] tree lexicon

Definition at line 1367 of file wchmm.c.

void build_wchmm ( WCHMM_INFO *  wchmm  ) 

Build a tree lexicon from given word dictionary and language model. This function is slow and only used when "-oldtree" option is specified in Julian. Julian without that option and Julius uses build_wchmm2() instead of this.

Parameters:
wchmm [i/o] lexicon tree

Definition at line 1491 of file wchmm.c.

Referenced by final_fusion(), and multigram_setup().

void build_wchmm2 ( WCHMM_INFO *  wchmm  ) 

Build a tree lexicon from given word dictionary and language model. This function does the same job as build_wchmm(), but it is much faster because finding of the longest matched word to an adding word is done by first sorting all the words in the dictoinary by their phoneme sequence order. This function will be used instead of build_wchmm() by default.

Parameters:
wchmm [i/o] lexicon tree

Definition at line 1682 of file wchmm.c.

Referenced by final_fusion(), and multigram_setup().

void print_wchmm_info ( WCHMM_INFO *  wchmm  ) 

Output some specifications of the tree lexicon (size etc.) to stdout.

Parameters:
wchmm [in] tree lexicon already built

Definition at line 1958 of file wchmm.c.

Referenced by print_info().

Generated on Tue Dec 26 12:53:49 2006 for Julian by  doxygen 1.5.0