Tomasz Sowa
463cec3283
fixed #2 : Procedures for reading an utf8 string incorrectly read some utf-8 characters.
...
Those characters were treated as invalid characters.
UTF8ToInt_FirstOctet incorrectly checked if the first octed is zero (after removing first bits).
This is a case only if the utf-8 character consists of two bytes. For 3 or 4 bytes
the first part can have all bits equal zero.
2021-05-07 15:53:19 +02:00
Tomasz Sowa
4ef8f5a884
template<size_t stack_size, size_t heap_block_size> changed to template<typename StreamType>
2021-03-15 20:09:23 +01:00
Tomasz Sowa
fac3a7eb71
reorganization in utf8
...
- utf8 auxiliary functions moved to utf8_private.h file
- in utf8.h are shown only functions available for consumers
- template functions has been moved to utf8_template.h (in utf8.h are only declarations)
utf8_template.h is included at the end of utf8.h
- functions which take std::ostream changed to template (the stream is a template argument now)
2021-03-15 19:34:51 +01:00
Tomasz Sowa
6c4a76baad
changed in utf8: functions taking WTextStream now take a template TextStreamBase
...
git-svn-id: svn://ttmath.org/publicrep/pikotools/trunk@1157 e52654a7-88a9-db11-a3e9-0013d4bc506e
2018-11-24 17:09:25 +00:00
Tomasz Sowa
da4ed9c1f8
added to utf8:
...
bool UTF8ToWide(const char * utf8, size_t utf8_len, WTextStream & res, bool clear = true, int mode = 1);
bool UTF8ToWide(const char * utf8, WTextStream & res, bool clear = true, int mode = 1);
bool UTF8ToWide(const std::string & utf8, WTextStream & res, bool clear = true, int mode = 1);
bool UTF8ToWide(std::istream & utf8, WTextStream & res, bool clear = true, int mode = 1);
void WideToUTF8(PT::WTextStream & buffer, std::string & utf8, bool clear = true, int mode = 1);
void WideToUTF8(PT::WTextStream & buffer, std::ostream & utf8, int mode = 1);
these functions need some testing yet
git-svn-id: svn://ttmath.org/publicrep/pikotools/trunk@1156 e52654a7-88a9-db11-a3e9-0013d4bc506e
2018-11-24 13:30:45 +00:00
Tomasz Sowa
d655c7c872
removed: svn:executable attribute from files
...
git-svn-id: svn://ttmath.org/publicrep/pikotools/trunk@985 e52654a7-88a9-db11-a3e9-0013d4bc506e
2014-11-01 17:11:00 +00:00
Tomasz Sowa
39717a4dd2
fixed: in static size_t WideToInt(const wchar_t * wide_string, size_t string_len, int & z, bool & correct)
...
we didn't test UTF8_CheckRange()
added: functions for converting from a wide string into an utf8 c-string:
bool WideToUTF8(const wchar_t * wide_string, size_t string_len, char * utf8, size_t utf8_len, size_t & utf8_written, int mode = 1);
bool WideToUTF8(const wchar_t * wide_string, char * utf8, size_t utf8_len, size_t & utf8_written, int mode = 1);
bool WideToUTF8(const std::wstring & wide_string, char * utf8, size_t utf8_len, size_t & utf8_written, int mode = 1);
bool WideToUTF8(const wchar_t * wide_string, size_t string_len, char * utf8, size_t utf8_len, int mode = 1);
bool WideToUTF8(const wchar_t * wide_string, char * utf8, size_t utf8_len, int mode = 1);
bool WideToUTF8(const std::wstring & wide_string, char * utf8, size_t utf8_len, int mode = 1);
git-svn-id: svn://ttmath.org/publicrep/pikotools/trunk@962 e52654a7-88a9-db11-a3e9-0013d4bc506e
2014-10-04 11:18:33 +00:00
Tomasz Sowa
d0ffdd336c
added: namespace PT, global Makefile file
...
git-svn-id: svn://ttmath.org/publicrep/pikotools/trunk@374 e52654a7-88a9-db11-a3e9-0013d4bc506e
2012-01-12 02:46:16 +00:00
Tomasz Sowa
8f9887d01a
moving utf8.h from ezc to pikotools
...
git-svn-id: svn://ttmath.org/publicrep/pikotools/trunk@366 e52654a7-88a9-db11-a3e9-0013d4bc506e
2012-01-11 11:15:20 +00:00
Tomasz Sowa