api2021 part I #4

tomasz.sowa · 2021-05-27T10:36:57+02:00

tomasz.sowa commented

2021-05-27 10:36:57 +02:00

merge some work from api2021 branch:

new Space struct (full json support)
PascalCase to snake_case in some places
CSVParser
MainOptionsParser
some tests

merge some work from api2021 branch: - new Space struct (full json support) - PascalCase to snake_case in some places - CSVParser - MainOptionsParser - some tests

tomasz.sowa added 67 commits 2021-05-27 10:36:59 +02:00

renamed: std::wstring Toa(...) -> std::wstring to_str(...) f771f12027

added: std::string to_str(...)

temporarily blocked for compilation: mainspaceparser, jsontospaceparser, spaceparser, spacetojson df88d4c0fc

added: typedef with char_type to TextStreamBase effe9be0a3

reorganization in utf8 fac3a7eb71

- utf8 auxiliary functions moved to utf8_private.h file
- in utf8.h are shown only functions available for consumers
- template functions has been moved to utf8_template.h (in utf8.h are only declarations)
  utf8_template.h is included at the end of utf8.h
- functions which take std::ostream changed to template (the stream is a template argument now)

make depend 7ccd435e2c

start working on a new version of Space struct - better support for JSON format f65d934e8c

now we have a correct model, some methods for setting/getting values and serialization to json
(no serialization to Space yet)

template<size_t stack_size, size_t heap_block_size> changed to template<typename StreamType> 4ef8f5a884

use PT::WideStreamToUTF8() instead of PT::WideToUTF8() for a moment bc9e4a3844

fixed: in serialize_json_double(): the buffer length was calculated incorrectly: sizeof(char) changed to sizeof(wchar_t) ba7fa1c195

added to Space struct:
 void set_null();
 void set_empty_string();
 void set_empty_wstring();
 void set_empty_table();
 void set_empty_object();
 void clear();

changed: JSONToSpaceParser can parse json string/file to the new Space format now 0c0f15ab8a

(some minor chars escaping not ready yet)

make depend 61291cf7ad

changed the way how the library is built 5b5a1dfbb6

- now there is only one Makefile in the root directory and we have only one pikotools.a lib file
- removed: mainparser

added to Space: child_spaces and a name of a Space 31f7bdb857

added to JSONToSpaceParser: possibility to parse the Space format 6e169f7650

renamed: ParseFile() -> ParseJSONFile()
added: ParseSpaceFile(), ParseJSON(), ParseSpace()

removed: old SpaceParser 961a02ab39

renamed: JSONToSpaceParser -> SpaceParser ba82ac7cbe

removed: SpaceToJson 1baf5042fe

added to Space: serialization do Space format db5e516564

fixed: when reading a non-escaped space token we should check whether a delimiter was escaped or not 4119461f8e

also a '#' was not tested

added some indentation when printing 'pretty Space format' 679b9e6173

changed the indentation algorithm when pretty printing the space format (no visual changes) 539faa4976

added a test whether we have reach the end of the input string 1c643a08b0

without this such an input json (incorrect json string as there is no {} characters):
"key": "value"
would be parsed correctly: "key" would be parsed and the rest would be skipped

make depend 0aeac12fa0

fixed: in Space: pointers 'name' and 'child_spaces' were not correctly initialized in cctors a2339eed34

added in Space:
- some methods for adding values to an object, such as:
  Space & Space::add(const std::wstring & field, bool val) (bool, short, int, long, long long etc.)
- methods for creating lists:
  void Space::to_list(std::list<std::string> & output_list, bool clear_list) const
  bool Space::to_list(const wchar_t * field, std::list<std::string> & output_list, bool clear_list) const
- methods for converting a value from an object field:
  bool Space::to_bool(const wchar_t * field, bool default_value) const
- methods for testing strings:
  bool Space::is_equal(const char * val) const
  bool Space::is_equal(const std::string & val) const
  bool Space::is_equal(const wchar_t * val) const
  bool Space::is_equal(const std::wstring & val) const
- methods to get the raw pointer to a value from an object, such as:
  bool * Space::get_bool(const wchar_t * field)
  float * Space::get_float(const wchar_t * field)
- methods for finding a child space (used in Space format only)
  Space * Space::find_child_space(const wchar_t * name)
  Space & Space::find_add_child_space(const wchar_t * name)

fixed: calculating buffer length in Space: sizeof(char) -> sizeof(wchar_t) a1e8f13f46

added to Space: 5d34db5fcf

- methods for testing a string value if a Space is a string or a table:
  bool has_value(const char * val) const;
  bool has_value(const std::string & val) const;
  bool has_value(const wchar_t * val) const;
  bool has_value(const std::wstring & val) const;
- methods for testing a string value in an object (testing a string or a table):
  bool has_value(const wchar_t * field, const char * val) const;
  bool has_value(const wchar_t * field, const std::string & val) const;
  bool has_value(const wchar_t * field, const wchar_t * val) const;
  bool has_value(const wchar_t * field, const std::wstring & val) const;
- methods for removing a child space:
  void remove_child_space(const wchar_t * name);
  void remove_child_space(const std::wstring & name);
  void remove_child_space(size_t index);

fix in SpaceParser when parsing json format: floating point values such as 1.123e+01 where not correctly parsed ('+' character was not correctly parsed) 297940ff7c

added to Space: some methods for searching a field with ignoring case:
Space * get_object_field_nc(const wchar_t * field)
Space * get_object_field_nc(const std::wstring & field);
const Space * get_object_field_nc(const wchar_t * field) const;
const Space * get_object_field_nc(const std::wstring & field) const;

fixed: Space::has_value(...) incorrectly compared strings 16f6bdb358

make depend f572250572

added support for morm::Model to Log: 96eedd9be9

Log & operator<<(morm::Model & model);
but we need some kind of a macro to allow this

fixed #2 : Procedures for reading an utf8 string incorrectly read some utf-8 characters. 463cec3283

Those characters were treated as invalid characters.

UTF8ToInt_FirstOctet incorrectly checked if the first octed is zero (after removing first bits).
This is a case only if the utf-8 character consists of two bytes. For 3 or 4 bytes
the first part can have all bits equal zero.

fixed #3 : CompareNoCase incorrectly returned that string1 is greater than string2 for some characters b055c46ae8

when converting from 'char' to 'int' we should first convert to 'unsigned char' and then to 'int'

make depend 127f26884e

moved all directories to src subdirectory 3984c29fbf

changes in convert/text functions 7abe4b340a

- changed function names: PascalCase to snake_case
- templates functions moved to a seperate file (text_private.h)
- as a public api only available functions with char/wchar_t/std::string/std::wstring
- ToLower(...) changed to to_lower_emplace(...), similar ToUpper(...) to to_upper_emplace(...)
- added functions:
  std::string to_lower(const std::string & str);
  std::string to_upper(const std::string & str);
  and with std::wstring too
- functions with postfix 'NoCase' changed to 'nc'

added macro PT_HAS_MORM in Log class where operator<<(morm::Model & model) is used adee7d134f

removed definition of CXX and CXXFLAGS from Makefile b3cd4d5f7f

make depend cd9e501f48

added 'tests' directory with tests for the pikotools library ce81670bb6

currently only tests for convert/text functions

added global Makefile 7bdc4d2fc7

added tests/tests to .gitignore d3dd74b3d4

added ./m to .gitignore 1260b303bc

added .editorconfig file with tab style and tab size information 3d6c4e27c0

https://editorconfig.org/

fixed in Space: set_empty_string(), set_empty_wstring(), set_empty_table() and set_empty_object() didn't clear its object 77d7bb5e64

if the same kind of object already existed

updated MainSpaceParser to the new Space format, changed api to snake case ac691bccb7

now we can:
- parse short options, those beginning with a hypnen '-'
- parse long options, those beginning with two hyphens '--'
- long options can have arguments in two forms:
  - either with an equal sign, e.g.: --opion-name=argument
  - or with a space, e.g: --option argument
    in the latter case we can have more than one argument, e.g: --option argument1 argument2
- parse non-option arguments, those after two hyphens to the end of a string, e.g: -- arg1 arg2

start creating tests for MainSpaceParser da6a36a205

changed the way of building in Makefiles fe82f63efb

make depend 91300bb245

added tests for MainSpaceParser a5c8833452

moved files: mainspaceparser/mainspaceparser.(h|cpp) -> mainoptions/mainoptionsparser.(h|cpp) 96e60c526f

MainSpaceParser class has been renamed to MainOptionsParser ad4e8078ae

make depend db93586c0e

removed *.a from .gitignore e0565813de

removed old not used variable from Space ae1d25d5f3

added CSVParser - a csv parser 0ea5497094

make depend 430822bad8

added move semantics to Space class 604b47db32

added methods:
Space(Space && space);
Space & operator=(Space && space);
void set(const Space & space);
void set(Space && space);
Space & add(const Space & space);
Space & add(Space && space);
Space & add(const wchar_t * field, const Space & space);
Space & add(const wchar_t * field, Space && space);
Space & add(const std::wstring & field, const Space & space);
Space & add(const std::wstring & field, Space && space);

fixed memory leak in CSVParser::parse() - a Space struct was allocated but never freed f8d24de386

fixed another memory leak in CSVParser::parse() (similar like before) 6f50626dda

namespace PT renamed to pt b574289054

changed utf8 functions: PascalCase to snake_case 59d4c9a9c8

fixed: SpaceParser was using space->set_empty_object() when parsing a space abeca010cc

and it cleared all values if not an empty object was provided to the set_space() method

removed: SpaceParser::SetSpace(...) methods, now ParseJSON/Space(...) methods take a space as an argument 82a21f6d85

removed from SpaceParser: SetDefault(), SkipEmpty(bool skip) and UseEscapeChar(bool escape) methods 6e4a0f68b3

changed: names of methods in SpaceParser: PascalCase to snake_case c11aa78335

changed the way how child_spaces are created in Space class 5ce36ea844

- removed child_spaces and name pointers
- now a table with child spaces is created under "child_spaces" object field
- a name of the child space is stored in "name" field of the child object

added methods for manipulating with child spaces:
TableType * find_child_space_table()
bool child_spaces_empty()
size_t child_spaces_size()

Space * find_child_space(size_t table_index)
Space & add_child_space(const wchar_t * space_name)
Space & add_child_space(const std::wstring & space_name)

std::wstring * find_child_space_name()
std::wstring get_child_space_name()
bool is_child_space_name(const wchar_t * name)

added additional methods:
size_t str_size()
size_t wstr_size()
size_t object_size()
size_t table_size()

fixed return value from Space::add_child_space() 49c2b478c0

tomasz.sowa merged commit 848cdf9c03 into master

2021-05-27 10:37:36 +02:00

tomasz.sowa referenced this issue from a commit

2021-05-27 10:37:38 +02:00

Merge pull request 'api2021 part I' (#4) from api2021 into master

tomasz.sowa referenced this issue from a commit

2021-06-18 18:54:48 +02:00

fixed in Space::find_child_space_const(...) - clang address sanitizer reports stack-use-after-scope

Sign in to join this conversation.