Commit Graph

62 Commits

Author SHA1 Message Date
Tomasz Sowa df32899581 move AcceptEncodingParser to utils 2022-02-02 18:34:22 +01:00
Tomasz Sowa 33e8df11c5 parse Accept-Language header to Request::accept_languages table 2022-02-02 18:20:32 +01:00
Tomasz Sowa 75daf37bbd improve AcceptBaseParser parsing algorithm
remove AcceptParser - not needed now, AcceptBaseParser can prepare a table now
2022-02-02 17:58:27 +01:00
Tomasz Sowa f7b5ac0dc8 change the way how winix answer is created
Now we can return ezc content and models serialized in the same json structure,
Xml and Csv are not implemented yet.
Ezc frames are returned in 'ezc_frames' field.
Main ezc stream is returned in 'main_stream' field.
Frame url parameter can take more than one frame (names separated by commas).
Honor Accept http header (AcceptParser).

Samples:
--------
http://domain.tld/dir/controller
returns html answer from the main ezc stream

http://domain.tld/dir/controller/container:raw
returns html answer from the main ezc stream (the same as above)

http://domain.tld/dir/controller/frame:abc
returns "abc" frame as html

http://domain.tld/dir/controller/container:json
returns all serialized models to json and no ezc streams

http://domain.tld/dir/controller/container:xml
returns all serialized models to xml and no ezc streams (not implemented yet)

http://domain.tld/dir/controller/container:json/frame:abc,xyz
returns all serialized models to json and two frames in 'ezc_frames' object

http://domain.tld/dir/controller/container:json/all_frames
returns all serialized models to json and all frames in 'ezc_frames' object

http://domain.tld/dir/controller/container:json/main_stream
returns all serialized models and the main ezc stream in 'main_stream' field

http://domain.tld/dir/controller/container:json/main_stream/all_frames
returns all serialized models to json, all frames and the main stream
2022-02-01 18:44:23 +01:00
Tomasz Sowa 227dd923d6 fix: correctly escape ezc stream to json/xml/csv 2022-01-28 15:54:11 +01:00
Tomasz Sowa d5ebb7ca12 changed the way how the request's answer is created,
now winix can return json, xml, csv from out_main_stream or from frames and json from models

removed from Request:
  bool send_bin_stream
  bool return_json
  bool return_info_only
  pt::Space info
  bool page_generated
  bool out_main_stream_use_html_filter
  bool out_streams_use_html_filter

added to Request:
  enum AnswerSource
  enum AnswerContainer
  AnswerSource answer_source
  AnswerContainer answer_container
  bool use_ezc_engine
  std::wstring frame
  bool send_all_frames
  bool use_html_filter

added to Config:
 // the name of the url parameter for returning all frames, e.g. https://domain.tld/mydir/myfunction/allframes
 // default: allframes
 std::wstring request_all_frames_parameter;

 // the name of the root element when serializing request answer to xml
 // default: winix
 std::wstring xml_root;

algorithm (the whole algorithm is described in core/request.h):
at the beginning of a request winix sets
   answer_source to models
   answer_container to text
   use_ezc_engine to true

next answer_container and use_ezc_engine can be changed in the following way:

1. winix will look for 'Accept' http header and depending on the header winix will set:
   (not implemented yet)

             Accept | answer_container | use_ezc_engine
	     ------------------------------------|-----------------
	     application/json | json             | false
	      application/xml | xml              | false
	              text/csv | csv              | false

2. next answer_container is set depending on 'container' url parameter
       container | answer_container
 ---------------------------------------------------------
     not present | don't change the value
            text | text
            json | json
             xml | xml
             csv | csv

use_ezc_engine is set depending on 'answer' url parameter:
          answer | use_ezc_engine
 ---------------------------------
     not present | don't change the value
            html | true
            data | false

if 'answer' is html then we take into account two more parameters:
 frame: frame_name (empty default) - if set then winix returns this specific frame
 allframes: (if present then winix returns all frames)
2021-10-13 01:27:14 +02:00
Tomasz Sowa c4b5565995 don't check for request->is_htmx_request for sending frames, check only "frame" url paremeter
added config parameter: request_frame_parameter, default: "frame"
2021-10-08 03:08:11 +02:00
Tomasz Sowa 9c5c74ba84 added: setting a correct mime type for static files - using magic library
added: std::wstring file_mime_type to ItemContent - a mime type for static file
added: Header (core/header.h) - there will be header names defined, at the moment only content_type
added: FuncionsBase::Finish() - it is called at the end when the winix finishes
2021-09-22 00:23:25 +02:00
Tomasz Sowa 26ed7b80be changed values for send_file_mode config parameter:
0 - winix will read the content of the file and send it back to the webserver
  1 - winix will use send_file_header header with a full path to the file
  2 - winix will use send_file_header header with a relative path to the file
2021-09-17 03:35:56 +02:00
Tomasz Sowa 55ac9a61ed added possibility to send static files to nginx via X-Accel-Redirect header
added to config:
int send_file_mode;
    // 0 - full path to a file in send_file_header header
    // 1 - relative path to a file in send_file_header (need http_send_file_relative_prefix set) (used for nginx)
std::wstring send_file_header;
    // default: X-SENDFILE
    // for Apache set: X-SENDFILE
    // for Lighttpd set: X-LIGHTTPD-send-file
    // for Nginx set: X-Accel-Redirect
std::wstring send_file_relative_prefix;
    // relative prefix used for sending static files if send_file_mode is 1
    // default: "upload-files-internal"
2021-09-15 20:28:34 +02:00
Tomasz Sowa 42dc43b036 added WINIX_MAKE_DATABASE_MIGRATION plugin message for making migrations from plugins 2021-08-11 22:18:14 +02:00
Tomasz Sowa c5c02d7f44 HtmlTextStream has now pt::Stream as a based class and uses pt::WTextStream as a buffer 2021-07-16 18:17:57 +02:00
Tomasz Sowa df3f04a951 added to misc: timespec_to_stream_with_unit() for printing times either is seconds or miliseconds 2021-06-30 16:30:11 +02:00
Tomasz Sowa 1d18b7fa12 - updated to the new pikotools api (child spaces were removed)
some plugins need to be fixed yet: ticket, gallery, group, menu
- added current user to default models as "user"
- renamed in User: super_user -> is_super_user, env -> admin_env, pass_hash_salted -> is_pass_hash_salted
- now Users class has a WinixModel as a base class
  some plugin calls have to be fixed yet
- added UserWrapper model with a pointer to User class
- removed from ItemContent: methods for accessing 'meta' and 'admin_meta', now ezc can iterate through Space classes
- fixed in env winix function: if there is "changeuser" parameter then we should only switch the user (not save anything)
2021-06-27 23:31:50 +02:00
Tomasz Sowa b6340a30d2 updated to the new pikotools api (changes in Log) 2021-06-24 21:17:03 +02:00
Tomasz Sowa 2f1cdcf379 added printing info how much time winix spent in the ezc engine
added to Request:
timespec timespec_req_stop;
timespec timespec_req_diff;
timespec timespec_ezc_engine_start;
timespec timespec_ezc_engine_stop;
2021-06-24 15:33:44 +02:00
Tomasz Sowa 8ab07d11b1 fixed: cur.request->function->fun didn't have a model connector set when a request started 2021-06-21 16:51:35 +02:00
Tomasz Sowa a1c0f6f7f0 added config option: add_header_cache_no_store_in_htmx_request
send "Cache-Control: no-store, max-age=0" http header if a request is made by htmx library (ajax)
if a webbrowser get a page from the cache then it will render just the last request without the whole html page (css, js, etc)
https://github.com/bigskysoftware/htmx/issues/497
2021-06-21 11:43:59 +02:00
Tomasz Sowa 79eda7abb0 - added to FunctionBase: bool register_default_models (default true)
if true then winix will add default models for ezc templates such as "request", "item", etc.
- some methods from templates/item.cpp moved to Item
2021-06-20 18:04:50 +02:00
Tomasz Sowa e7c7324058 ItemContent::user_name(EzcEnv & env) changed to ItemContent::user(morm::ModelWrapper ** model_wrapper) and we return a User class
ItemContent::group_name(EzcEnv & env) changed to ItemContent::group(morm::ModelWrapper ** model_wrapper) and we return a Group class
added ItemContent::display_user_name(EzcEnv & env)
some functions in templates/item.cpp moved to Item
2021-06-19 23:48:12 +02:00
Tomasz Sowa ec94dff7d7 some ezc functions from templates/item.cpp moved to Item and ItemContent
methods HasAccess() HasReadAccess() and similar moved from System to Item and ItemContent
2021-06-18 19:18:13 +02:00
Tomasz Sowa ebd791a256 changed: now Request class is a model class - we have fields() method there to map fields for ezc templates (currently only dir_tab)
removed: [dir_tab] ezc statement
changed: Ls winix function loads its own dir_tab container (beforehand it was loaded by [dir_tab] ezc statement)
         it's available by [child_dirs] name
2021-06-17 21:44:29 +02:00
Tomasz Sowa ab89ffe096 fixed in Item: void Item::link(Ezc::FunInfo<HtmlTextStream> & env) now returns a correct directory for the item (before the current directory was returned)
added to Item: void Item::dir_link(Ezc::FunInfo<HtmlTextStream> & env)
added to WinixModel:
  Dirs * get_dirs();
  Mounts * get_mounts();
  Users * get_users();
  Groups * get_groups();
  SLog * get_session_logger(); // not finished yet
2021-06-16 23:42:23 +02:00
Tomasz Sowa 6dddc5e948 renamed: WinixModel -> WinixModelDeprecated (this class will be removed)
added: WinixModel (models/winixmodel.h|cpp) - a class inheriting from morm::WinixModel, we have methods such as: get_config(), get_logger(), get_request()
       and this class will be a base class for our models
added: WinixModelConnector (models/winixmodelconnector.h|cpp) - a class inheriting from morm::WinixModelConnector
       this connector we are using instead of morm::ModelConnector - there are pointers to winix objects there (config, request, log)
added to Request: Ezc::Models models
removed from TemplatesFunctions: Ezc::Models ezc_models;
changed: ImgCrop winix functions is using its own item_tab vector now (not finished yet)
added: Item::is(), Item::link(), ItemContent::print_content()
2021-06-16 18:07:44 +02:00
Tomasz Sowa 22de3322ae - added to Request: is_htmx_request (bool) - true if the request is made by htmx library (ajax)
- commented out using mount_page_arg_is "subject" and "info" from fun_cat.html and fun_run.html - they will be removed soon
- some improvements in fun_reply.html, fun_showthreads.html and fun_thread.html (scrolling)
2021-05-31 10:58:34 +02:00
Tomasz Sowa e6fae4598c updated to the new pikotools api: [ezc out] and [out] changed to [ezc frame] or just [frame]
- when returning json streams changed "out" field to "frames"
- renamed config parameter: allow_ezc_out_in_executable_items -> allow_ezc_frames_in_executable_items
- added basic support for ajax requests in thread plugin
2021-05-28 18:36:19 +02:00
Tomasz Sowa ba331dea4a added basic support for htmx (ajax)
- if there is HX-Request header present we sent only a part of the whole html
- we return only specific stream defined by [out ...] ezc statement
- the name of the stream is passed in the 'frame' parameter
  (if not present then 'content' is assumed)
- added ezc function: winix_is_htmx_request
2021-05-27 19:36:04 +02:00
Tomasz Sowa fe67b5d72a updated to the new pikotools api: snake_case names of methods from SpaceParser 2021-05-21 04:51:58 +02:00
Tomasz Sowa 27720afaf2 updated to the new pikotools api: SpaceParser::SetSpace(...) methods have been removed 2021-05-21 01:37:42 +02:00
Tomasz Sowa 8bb585d97d updated to the new pikotools api: changed utf8 functions PascalCase to snake_case 2021-05-21 00:41:27 +02:00
Tomasz Sowa e48a28a5c8 namespace PT renamed to pt 2021-05-20 20:59:12 +02:00
Tomasz Sowa a94e09f0aa added support for Model migrations
now we have a table core.migration and each model (User, Group, Item, ItemContent and a new Migration) have its own
row in the table with a version number

added to config:
db_make_migration_if_needed and db_stop_if_migration_fails (need description yet)
2021-05-14 03:31:29 +02:00
Tomasz Sowa f1af7e2eeb updated to the current pikotools api from branch api2021 2021-05-11 22:20:46 +02:00
Tomasz Sowa 4277f90bad start working on User and Group classes
- User and Group has been moved to 'models' directory
- removed UserPass struct (passwords fields were put to User struct)
not working yet, we need support for binary blobs in morm
2021-04-30 01:34:48 +02:00
Tomasz Sowa ccda2bc2fd added option do_migration_to_winix_fullmorm do the config (only temporarily)
if true then we make Item::do_migration() and exit
2021-04-14 14:13:47 +02:00
Tomasz Sowa 486067a4b2 clang thread sanitizer reported a data race in App::Start()
synchro.was_stop_signal was read without locking

==================
WARNING: ThreadSanitizer: data race (pid=13831)
  Write of size 1 at 0x000000dc4d18 by thread T1 (mutexes: write M44):
    #0 Winix::App::SpecialThreadForSignals(void*) /disk2Tc/home/tomek/roboczy/prog/winix/winixd/core/app.cpp:2226:31 (winix.so+0x1f7465)

  Previous read of size 1 at 0x000000dc4d18 by main thread:
    #0 Winix::App::Start() /disk2Tc/home/tomek/roboczy/prog/winix/winixd/core/app.cpp:651:18 (winix.so+0x1ee9af)
    #1 main /disk2Tc/home/tomek/roboczy/prog/winix/winixd/main/main.cpp:222:6 (winix+0x2d74c1)

  Location is global 'Winix::app' of size 65840 at 0x000000db5908 (winix+0x000000dc4d18)

  Mutex M44 (0x000000dc4d10) created at:
    #0 pthread_mutex_lock /disk2Tb/usr/src/contrib/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_common_interceptors.inc:4153:3 (winix+0x2ac204)
    #1 Winix::Synchro::Lock() /disk2Tc/home/tomek/roboczy/prog/winix/winixd/core/synchro.cpp:68:12 (winix.so+0x2e5822)
    #2 Winix::Lock::Lock(Winix::Synchro*) /disk2Tc/home/tomek/roboczy/prog/winix/winixd/core/lock.cpp:53:11 (winix.so+0x295522)
    #3 Winix::FileLog::get_local_date(PT::Date const&) /disk2Tc/home/tomek/roboczy/prog/winix/winixd/core/filelog.cpp:98:8 (winix.so+0x261183)
    #4 Winix::Log::PrintDate(PT::Date const&) /disk2Tc/home/tomek/roboczy/prog/winix/winixd/core/log.cpp:80:30 (winix.so+0x295a21)
    #5 Winix::LogInfo(Winix::Log&, Winix::LogManipulators, char const*, bool, char const*) /disk2Tc/home/tomek/roboczy/prog/winix/winixd/main/main.cpp:104:6 (winix+0x2d69ec)
    #6 main /disk2Tc/home/tomek/roboczy/prog/winix/winixd/main/main.cpp:190:2 (winix+0x2d7363)

  Thread T1 (tid=100187, running) created by main thread at:
    #0 pthread_create /disk2Tb/usr/src/contrib/llvm-project/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:962:3 (winix+0x295bb3)
    #1 Winix::App::StartThreads() /disk2Tc/home/tomek/roboczy/prog/winix/winixd/core/app.cpp:2246:2 (winix.so+0x1f7549)
    #2 main /disk2Tc/home/tomek/roboczy/prog/winix/winixd/main/main.cpp:212:6 (winix+0x2d743e)

SUMMARY: ThreadSanitizer: data race /disk2Tc/home/tomek/roboczy/prog/winix/winixd/core/app.cpp:2226:31 in Winix::App::SpecialThreadForSignals(void*)
==================
2021-04-10 17:14:51 +02:00
Tomasz Sowa 35e10ed469 updated to the new Pikotools api (new Space struct) 2021-04-09 17:50:58 +02:00
Tomasz Sowa bbc0a67153 added support for new morm api
- field(...) methods from Model take now FT structure instead of insertable/updatable/primary_key boolean flags
- table name is set in a prepare_table() method now
- added meta and admin_meta from ItemContent (morm has a field() method with PT::Space argument now)
2021-03-11 18:48:18 +01:00
Tomasz Sowa 32e93a04c5 Item class has been moved to a new directory 'models', a new class has been added: ItemContent
and same fields from Item were moved to ItemContent

Item
- id
- parent_id
- type (file, dir, symlink)
- url
- subject
- template (html template)
- sort_index
- content_id

ItemContent
- id
- ref -> references (renamed)
- user_id
- modification_user_id
- group_id
- privileges
- date_creation
- date_modification
- guest_name
- link_to
- link_redirect
- file_path
- file_fs
- file_type
- file_size
- has_thumb -> file_has_thumb (renamed)
- hash -> file_hash (renamed)
- hash_type -> file_hash_type (renamed)
- content -> content_raw (renamed)
- content_type -> content_raw_type (renamed)
- content_parsed
- content_parsed_type
- meta
- ameta -> meta_admin (renamed)
- modify_index (removed)

WIP: #4
2021-02-24 01:19:47 +01:00
Tomasz Sowa 177e58da4e changed: if there is no body we do not parse application/json to space
git-svn-id: svn://ttmath.org/publicrep/winix/trunk@1183 e52654a7-88a9-db11-a3e9-0013d4bc506e
2019-03-25 00:51:51 +00:00
Tomasz Sowa 4c2efc08fd fixed: permissions to symlinks and directories with redirect flag were incorrectly checked
(there was no session set and the request was treated the same as from a not logged user)
fixed: in BaseThread there was used 'log' in the main thread (this logger is only for the other thread)
added: in BaseThread we have a main_log now - logger which puts to the main log buffer from the main thread




git-svn-id: svn://ttmath.org/publicrep/winix/trunk@1182 e52654a7-88a9-db11-a3e9-0013d4bc506e
2019-03-19 18:08:09 +00:00
Tomasz Sowa 8895f0dcac - json_generic_serializer was not set when there was a redirect
- if there is a redirect and the json return answer is requested
  then we send 'redirect_to' field in 'info' space




git-svn-id: svn://ttmath.org/publicrep/winix/branches/0.7.x@1169 e52654a7-88a9-db11-a3e9-0013d4bc506e
2019-02-19 10:36:20 +00:00
Tomasz Sowa b90445de4a updated to the new pikotools api - we have a logger for morm and ezc
pikotools has now Log and FileLog classes
which are base classes for winix Log and FileLog classes




git-svn-id: svn://ttmath.org/publicrep/winix/branches/0.7.x@1151 e52654a7-88a9-db11-a3e9-0013d4bc506e
2018-11-23 17:53:43 +00:00
Tomasz Sowa 491dd27ebf some work in the new api:
- log_level and save_each_line moved from Log to FileLog
- other fixes in loggers
- DirContainer from Dirs and LoadAvg had no dependencies set




git-svn-id: svn://ttmath.org/publicrep/winix/branches/0.7.x@1148 e52654a7-88a9-db11-a3e9-0013d4bc506e
2018-11-21 22:53:24 +00:00
Tomasz Sowa 89d303f375 some work on dependency injection
git-svn-id: svn://ttmath.org/publicrep/winix/branches/0.7.x@1147 e52654a7-88a9-db11-a3e9-0013d4bc506e
2018-11-21 17:51:15 +00:00
Tomasz Sowa a2ffc1e81c start working on 0.7.x branch
- added FileLog which stores content to the file log
- now Log is only a wrapper - it puts messages to the local buffer and when logsave is used then the buffer is put to FileLog
- new base classes:
  WinixBase (Log, Config*, Synchro*)
  WinixModel : public WinixBase (morm::ModelConnector*, Plugin*)
  WinixSystem : public WinixModel (System*)
  WinixRequest : public WinixSystem (SLog, Cur*)
- singletons: log, slog, plugin are depracated - now references to them are in base classses (WinixBase, WinixModel)
- DbBase,  DbConn and Db are depracated - now we are using Morm project (in WinixModel there is a model_connector pointer)
  each thread will have its own ModelConnector





git-svn-id: svn://ttmath.org/publicrep/winix/branches/0.7.x@1146 e52654a7-88a9-db11-a3e9-0013d4bc506e
2018-11-21 11:03:53 +00:00
Tomasz Sowa 919c2f62f5 using PT::IsSubString* methods from pikotools
git-svn-id: svn://ttmath.org/publicrep/winix/trunk@1138 e52654a7-88a9-db11-a3e9-0013d4bc506e
2018-11-01 00:46:44 +00:00
Tomasz Sowa 350fc551e5 fixed: a default winix function was selected before the mount point was calculated
also a mount point should be cleared (set to an empty mount point) at the
       end of a request




git-svn-id: svn://ttmath.org/publicrep/winix/trunk@1117 e52654a7-88a9-db11-a3e9-0013d4bc506e
2018-07-02 12:02:19 +00:00
Tomasz Sowa 08123fe6ac changed the algorithm how sessions work:
- if the IP is banned or there is no a winix function then we set a temporary session
- else
  if there is a session's cookie sent by the client then:
   - if the cookie is a correct session's cookie then we set the session from the cookie
   - or if the cookie is not a correct session's cookie (e.g. session expired) and the winix function
     requires a cookie then we set a new session
   - or if there is no cookie sent then if a winix function requires a session we create a new session

  if there was an error creating a new session or event counters reach a ban limit then a temporary session will be used





git-svn-id: svn://ttmath.org/publicrep/winix/trunk@1115 e52654a7-88a9-db11-a3e9-0013d4bc506e
2018-06-20 15:00:02 +00:00
Tomasz Sowa 35b93b1655 changed: now we are parsing directories and functions before checking a session
added:   need_session to FunctionBase (true if the functions requires a session)
         default: true
         



git-svn-id: svn://ttmath.org/publicrep/winix/trunk@1114 e52654a7-88a9-db11-a3e9-0013d4bc506e
2018-06-19 13:52:08 +00:00