mystrtok.c File Reference

Extract tokens from strings, with quotation handling. More...

#include <sent/stddefs.h>

Include dependency graph for mystrtok.c:

Go to the source code of this file.

Defines
#define	ISTOKEN(A)   strchr(delim, A)
	Abbreviation macro to find if A is included in string delim.
Functions
char *	mystrtok_quotation (char *str, char *delim, int left_paren, int right_paren, int mode)
char *	mystrtok_quote (char *str, char *delim)
char *	mystrtok (char *str, char *delim)
char *	mystrtok_movetonext (char *str, char *delim)

---

Detailed Description

Extract tokens from strings, with quotation handling.

Author:

Akinobu LEE

Date:

Thu Feb 17 16:31:39 2005

When extracting tokens from strings, the part enclosed by the specified braces are forced to be treated as a single token.

Revision

1.1.1.1

Definition in file mystrtok.c.

---

Function Documentation

char* mystrtok_quotation (  char *  str, char *  delim, int  left_paren, int  right_paren, int  mode )

Generic function to extract tokens from strings, with quotation handling. The usage is as the same as strtok. Parameters: str  [i/o] source string, or NULL when this is a continuous call from previous call. Will be truncated in this function. delim  [in] string to specify the delimiters. left_paren  [in] left brace right_paren  [in] right brace mode  [in] if 1, just move to the beginning of next token Returns: pointer to the next extracted token. Definition at line 50 of file mystrtok.c. Referenced by mystrtok(), mystrtok_movetonext(), mystrtok_quote(), and voca_load_htkdict_line().

char* mystrtok_quote (  char *  str, char *  delim )

Extract tokens considering quotation by double quotation mark. Parameters: str  [i/o] source string, will be truncated. delim  [in] string of all token delimiters Returns: pointer to the next extracted token, or NULL when no token found. Definition at line 104 of file mystrtok.c. Referenced by read_token().

char* mystrtok (  char *  str, char *  delim )

Extract tokens, not considering quotation, just as the same as strtok. Parameters: str  [i/o] source string, will be truncated. delim  [in] string of all token delimiters Returns: pointer to the next extracted token, or NULL when no token found. Definition at line 118 of file mystrtok.c. Referenced by voca_load_htkdict_line().

char* mystrtok_movetonext (  char *  str, char *  delim )

Just move to the beginning of the next token, without modifying the str. Parameters: str  [i/o] source string, will be truncated. delim  [in] string of all token delimiters Returns: pointer to the next extracted token, or NULL when no token found. Definition at line 132 of file mystrtok.c. Referenced by voca_load_htkdict_line().

---