libsent/src/ngram/ngram_write_bin.c File Reference

Write a whole N-gram data to a file in binary format. More...

#include <sent/stddefs.h>
#include <sent/ngram2.h>

Go to the source code of this file.

Defines
#define	wrt(A, B, C, D)   if (wrtfunc(A,B,C,D) == FALSE) return FALSE
Functions
void	reset_wrt_counter ()
int	get_wrt_counter ()
static boolean	wrtfunc (FILE *fp, void *buf, size_t unitbyte, int unitnum)
	Binary write function, with byte swapping if needed.
static boolean	write_header (FILE *fp, char *str)
	Write header information, with identifier string.
boolean	ngram_write_bin (FILE *fp, NGRAM_INFO *ndata, char *headerstr)
	Write a whole N-gram data in binary format.
Variables
static boolean	need_swap
	TRUE if need byte swap.
static int	count

---

Detailed Description

Write a whole N-gram data to a file in binary format.

From 3.5, internal format of binary N-gram has changed for using machine-dependent natural byte order (previously fixed to big endian), 24bit index and 2-gram backoff compression. So, binary N-gram generated by mkbingram of 3.5 and later will not work on 3.4.2 and earlier versions.

There is full upward- and cross-machine compatibility in 3.5. Old binary N-gram files still can be read directly, in which case the conversion to 24bit index will performed just after model has been read. Byte order will also considered by header information, so binary N-gram still can be used among different machines.

Author:

Akinobu LEE

Date:

Wed Feb 16 17:23:16 2005

Revision

1.1.1.1

Definition in file ngram_write_bin.c.

---

Function Documentation

static boolean wrtfunc	(	FILE *	fp,
		void *	buf,
		size_t	unitbyte,
		int	unitnum
	)			[static]

Binary write function, with byte swapping if needed.

Parameters:

	fp	[in] file pointer
	buf	[in] data buffer to write
	unitbyte	[in] unit size in bytes
	unitnum	[in] number of unit to write

Definition at line 82 of file ngram_write_bin.c.

static boolean write_header	(	FILE *	fp,
		char *	str
	)			[static]

Write header information, with identifier string.

Parameters:

	fp	[in] file pointer
	str	[in] user header string (any string within BINGRAM_HDSIZE bytes is allowed)
	version	[in] file format version id

Definition at line 107 of file ngram_write_bin.c.

Referenced by ngram_write_bin().

boolean ngram_write_bin	(	FILE *	fp,
		NGRAM_INFO *	ndata,
		char *	headerstr
	)

Write a whole N-gram data in binary format.

Parameters:

	fp	[in] file pointer
	ndata	[in] N-gram data to write
	headerstr	[in] user header string

Returns:

TRUE on success, FALSE on failure

Definition at line 135 of file ngram_write_bin.c.

---