Browse Source

release v0

Harlan Iverson 2 năm trước cách đây
commit
784ab587d0

+ 2 - 0
.gitignore

@@ -0,0 +1,2 @@
+__pycache__
+*.pyc

+ 661 - 0
LICENSE-AGPL3.txt

@@ -0,0 +1,661 @@
+                    GNU AFFERO GENERAL PUBLIC LICENSE
+                       Version 3, 19 November 2007
+
+ Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
+ Everyone is permitted to copy and distribute verbatim copies
+ of this license document, but changing it is not allowed.
+
+                            Preamble
+
+  The GNU Affero General Public License is a free, copyleft license for
+software and other kinds of works, specifically designed to ensure
+cooperation with the community in the case of network server software.
+
+  The licenses for most software and other practical works are designed
+to take away your freedom to share and change the works.  By contrast,
+our General Public Licenses are intended to guarantee your freedom to
+share and change all versions of a program--to make sure it remains free
+software for all its users.
+
+  When we speak of free software, we are referring to freedom, not
+price.  Our General Public Licenses are designed to make sure that you
+have the freedom to distribute copies of free software (and charge for
+them if you wish), that you receive source code or can get it if you
+want it, that you can change the software or use pieces of it in new
+free programs, and that you know you can do these things.
+
+  Developers that use our General Public Licenses protect your rights
+with two steps: (1) assert copyright on the software, and (2) offer
+you this License which gives you legal permission to copy, distribute
+and/or modify the software.
+
+  A secondary benefit of defending all users' freedom is that
+improvements made in alternate versions of the program, if they
+receive widespread use, become available for other developers to
+incorporate.  Many developers of free software are heartened and
+encouraged by the resulting cooperation.  However, in the case of
+software used on network servers, this result may fail to come about.
+The GNU General Public License permits making a modified version and
+letting the public access it on a server without ever releasing its
+source code to the public.
+
+  The GNU Affero General Public License is designed specifically to
+ensure that, in such cases, the modified source code becomes available
+to the community.  It requires the operator of a network server to
+provide the source code of the modified version running there to the
+users of that server.  Therefore, public use of a modified version, on
+a publicly accessible server, gives the public access to the source
+code of the modified version.
+
+  An older license, called the Affero General Public License and
+published by Affero, was designed to accomplish similar goals.  This is
+a different license, not a version of the Affero GPL, but Affero has
+released a new version of the Affero GPL which permits relicensing under
+this license.
+
+  The precise terms and conditions for copying, distribution and
+modification follow.
+
+                       TERMS AND CONDITIONS
+
+  0. Definitions.
+
+  "This License" refers to version 3 of the GNU Affero General Public License.
+
+  "Copyright" also means copyright-like laws that apply to other kinds of
+works, such as semiconductor masks.
+
+  "The Program" refers to any copyrightable work licensed under this
+License.  Each licensee is addressed as "you".  "Licensees" and
+"recipients" may be individuals or organizations.
+
+  To "modify" a work means to copy from or adapt all or part of the work
+in a fashion requiring copyright permission, other than the making of an
+exact copy.  The resulting work is called a "modified version" of the
+earlier work or a work "based on" the earlier work.
+
+  A "covered work" means either the unmodified Program or a work based
+on the Program.
+
+  To "propagate" a work means to do anything with it that, without
+permission, would make you directly or secondarily liable for
+infringement under applicable copyright law, except executing it on a
+computer or modifying a private copy.  Propagation includes copying,
+distribution (with or without modification), making available to the
+public, and in some countries other activities as well.
+
+  To "convey" a work means any kind of propagation that enables other
+parties to make or receive copies.  Mere interaction with a user through
+a computer network, with no transfer of a copy, is not conveying.
+
+  An interactive user interface displays "Appropriate Legal Notices"
+to the extent that it includes a convenient and prominently visible
+feature that (1) displays an appropriate copyright notice, and (2)
+tells the user that there is no warranty for the work (except to the
+extent that warranties are provided), that licensees may convey the
+work under this License, and how to view a copy of this License.  If
+the interface presents a list of user commands or options, such as a
+menu, a prominent item in the list meets this criterion.
+
+  1. Source Code.
+
+  The "source code" for a work means the preferred form of the work
+for making modifications to it.  "Object code" means any non-source
+form of a work.
+
+  A "Standard Interface" means an interface that either is an official
+standard defined by a recognized standards body, or, in the case of
+interfaces specified for a particular programming language, one that
+is widely used among developers working in that language.
+
+  The "System Libraries" of an executable work include anything, other
+than the work as a whole, that (a) is included in the normal form of
+packaging a Major Component, but which is not part of that Major
+Component, and (b) serves only to enable use of the work with that
+Major Component, or to implement a Standard Interface for which an
+implementation is available to the public in source code form.  A
+"Major Component", in this context, means a major essential component
+(kernel, window system, and so on) of the specific operating system
+(if any) on which the executable work runs, or a compiler used to
+produce the work, or an object code interpreter used to run it.
+
+  The "Corresponding Source" for a work in object code form means all
+the source code needed to generate, install, and (for an executable
+work) run the object code and to modify the work, including scripts to
+control those activities.  However, it does not include the work's
+System Libraries, or general-purpose tools or generally available free
+programs which are used unmodified in performing those activities but
+which are not part of the work.  For example, Corresponding Source
+includes interface definition files associated with source files for
+the work, and the source code for shared libraries and dynamically
+linked subprograms that the work is specifically designed to require,
+such as by intimate data communication or control flow between those
+subprograms and other parts of the work.
+
+  The Corresponding Source need not include anything that users
+can regenerate automatically from other parts of the Corresponding
+Source.
+
+  The Corresponding Source for a work in source code form is that
+same work.
+
+  2. Basic Permissions.
+
+  All rights granted under this License are granted for the term of
+copyright on the Program, and are irrevocable provided the stated
+conditions are met.  This License explicitly affirms your unlimited
+permission to run the unmodified Program.  The output from running a
+covered work is covered by this License only if the output, given its
+content, constitutes a covered work.  This License acknowledges your
+rights of fair use or other equivalent, as provided by copyright law.
+
+  You may make, run and propagate covered works that you do not
+convey, without conditions so long as your license otherwise remains
+in force.  You may convey covered works to others for the sole purpose
+of having them make modifications exclusively for you, or provide you
+with facilities for running those works, provided that you comply with
+the terms of this License in conveying all material for which you do
+not control copyright.  Those thus making or running the covered works
+for you must do so exclusively on your behalf, under your direction
+and control, on terms that prohibit them from making any copies of
+your copyrighted material outside their relationship with you.
+
+  Conveying under any other circumstances is permitted solely under
+the conditions stated below.  Sublicensing is not allowed; section 10
+makes it unnecessary.
+
+  3. Protecting Users' Legal Rights From Anti-Circumvention Law.
+
+  No covered work shall be deemed part of an effective technological
+measure under any applicable law fulfilling obligations under article
+11 of the WIPO copyright treaty adopted on 20 December 1996, or
+similar laws prohibiting or restricting circumvention of such
+measures.
+
+  When you convey a covered work, you waive any legal power to forbid
+circumvention of technological measures to the extent such circumvention
+is effected by exercising rights under this License with respect to
+the covered work, and you disclaim any intention to limit operation or
+modification of the work as a means of enforcing, against the work's
+users, your or third parties' legal rights to forbid circumvention of
+technological measures.
+
+  4. Conveying Verbatim Copies.
+
+  You may convey verbatim copies of the Program's source code as you
+receive it, in any medium, provided that you conspicuously and
+appropriately publish on each copy an appropriate copyright notice;
+keep intact all notices stating that this License and any
+non-permissive terms added in accord with section 7 apply to the code;
+keep intact all notices of the absence of any warranty; and give all
+recipients a copy of this License along with the Program.
+
+  You may charge any price or no price for each copy that you convey,
+and you may offer support or warranty protection for a fee.
+
+  5. Conveying Modified Source Versions.
+
+  You may convey a work based on the Program, or the modifications to
+produce it from the Program, in the form of source code under the
+terms of section 4, provided that you also meet all of these conditions:
+
+    a) The work must carry prominent notices stating that you modified
+    it, and giving a relevant date.
+
+    b) The work must carry prominent notices stating that it is
+    released under this License and any conditions added under section
+    7.  This requirement modifies the requirement in section 4 to
+    "keep intact all notices".
+
+    c) You must license the entire work, as a whole, under this
+    License to anyone who comes into possession of a copy.  This
+    License will therefore apply, along with any applicable section 7
+    additional terms, to the whole of the work, and all its parts,
+    regardless of how they are packaged.  This License gives no
+    permission to license the work in any other way, but it does not
+    invalidate such permission if you have separately received it.
+
+    d) If the work has interactive user interfaces, each must display
+    Appropriate Legal Notices; however, if the Program has interactive
+    interfaces that do not display Appropriate Legal Notices, your
+    work need not make them do so.
+
+  A compilation of a covered work with other separate and independent
+works, which are not by their nature extensions of the covered work,
+and which are not combined with it such as to form a larger program,
+in or on a volume of a storage or distribution medium, is called an
+"aggregate" if the compilation and its resulting copyright are not
+used to limit the access or legal rights of the compilation's users
+beyond what the individual works permit.  Inclusion of a covered work
+in an aggregate does not cause this License to apply to the other
+parts of the aggregate.
+
+  6. Conveying Non-Source Forms.
+
+  You may convey a covered work in object code form under the terms
+of sections 4 and 5, provided that you also convey the
+machine-readable Corresponding Source under the terms of this License,
+in one of these ways:
+
+    a) Convey the object code in, or embodied in, a physical product
+    (including a physical distribution medium), accompanied by the
+    Corresponding Source fixed on a durable physical medium
+    customarily used for software interchange.
+
+    b) Convey the object code in, or embodied in, a physical product
+    (including a physical distribution medium), accompanied by a
+    written offer, valid for at least three years and valid for as
+    long as you offer spare parts or customer support for that product
+    model, to give anyone who possesses the object code either (1) a
+    copy of the Corresponding Source for all the software in the
+    product that is covered by this License, on a durable physical
+    medium customarily used for software interchange, for a price no
+    more than your reasonable cost of physically performing this
+    conveying of source, or (2) access to copy the
+    Corresponding Source from a network server at no charge.
+
+    c) Convey individual copies of the object code with a copy of the
+    written offer to provide the Corresponding Source.  This
+    alternative is allowed only occasionally and noncommercially, and
+    only if you received the object code with such an offer, in accord
+    with subsection 6b.
+
+    d) Convey the object code by offering access from a designated
+    place (gratis or for a charge), and offer equivalent access to the
+    Corresponding Source in the same way through the same place at no
+    further charge.  You need not require recipients to copy the
+    Corresponding Source along with the object code.  If the place to
+    copy the object code is a network server, the Corresponding Source
+    may be on a different server (operated by you or a third party)
+    that supports equivalent copying facilities, provided you maintain
+    clear directions next to the object code saying where to find the
+    Corresponding Source.  Regardless of what server hosts the
+    Corresponding Source, you remain obligated to ensure that it is
+    available for as long as needed to satisfy these requirements.
+
+    e) Convey the object code using peer-to-peer transmission, provided
+    you inform other peers where the object code and Corresponding
+    Source of the work are being offered to the general public at no
+    charge under subsection 6d.
+
+  A separable portion of the object code, whose source code is excluded
+from the Corresponding Source as a System Library, need not be
+included in conveying the object code work.
+
+  A "User Product" is either (1) a "consumer product", which means any
+tangible personal property which is normally used for personal, family,
+or household purposes, or (2) anything designed or sold for incorporation
+into a dwelling.  In determining whether a product is a consumer product,
+doubtful cases shall be resolved in favor of coverage.  For a particular
+product received by a particular user, "normally used" refers to a
+typical or common use of that class of product, regardless of the status
+of the particular user or of the way in which the particular user
+actually uses, or expects or is expected to use, the product.  A product
+is a consumer product regardless of whether the product has substantial
+commercial, industrial or non-consumer uses, unless such uses represent
+the only significant mode of use of the product.
+
+  "Installation Information" for a User Product means any methods,
+procedures, authorization keys, or other information required to install
+and execute modified versions of a covered work in that User Product from
+a modified version of its Corresponding Source.  The information must
+suffice to ensure that the continued functioning of the modified object
+code is in no case prevented or interfered with solely because
+modification has been made.
+
+  If you convey an object code work under this section in, or with, or
+specifically for use in, a User Product, and the conveying occurs as
+part of a transaction in which the right of possession and use of the
+User Product is transferred to the recipient in perpetuity or for a
+fixed term (regardless of how the transaction is characterized), the
+Corresponding Source conveyed under this section must be accompanied
+by the Installation Information.  But this requirement does not apply
+if neither you nor any third party retains the ability to install
+modified object code on the User Product (for example, the work has
+been installed in ROM).
+
+  The requirement to provide Installation Information does not include a
+requirement to continue to provide support service, warranty, or updates
+for a work that has been modified or installed by the recipient, or for
+the User Product in which it has been modified or installed.  Access to a
+network may be denied when the modification itself materially and
+adversely affects the operation of the network or violates the rules and
+protocols for communication across the network.
+
+  Corresponding Source conveyed, and Installation Information provided,
+in accord with this section must be in a format that is publicly
+documented (and with an implementation available to the public in
+source code form), and must require no special password or key for
+unpacking, reading or copying.
+
+  7. Additional Terms.
+
+  "Additional permissions" are terms that supplement the terms of this
+License by making exceptions from one or more of its conditions.
+Additional permissions that are applicable to the entire Program shall
+be treated as though they were included in this License, to the extent
+that they are valid under applicable law.  If additional permissions
+apply only to part of the Program, that part may be used separately
+under those permissions, but the entire Program remains governed by
+this License without regard to the additional permissions.
+
+  When you convey a copy of a covered work, you may at your option
+remove any additional permissions from that copy, or from any part of
+it.  (Additional permissions may be written to require their own
+removal in certain cases when you modify the work.)  You may place
+additional permissions on material, added by you to a covered work,
+for which you have or can give appropriate copyright permission.
+
+  Notwithstanding any other provision of this License, for material you
+add to a covered work, you may (if authorized by the copyright holders of
+that material) supplement the terms of this License with terms:
+
+    a) Disclaiming warranty or limiting liability differently from the
+    terms of sections 15 and 16 of this License; or
+
+    b) Requiring preservation of specified reasonable legal notices or
+    author attributions in that material or in the Appropriate Legal
+    Notices displayed by works containing it; or
+
+    c) Prohibiting misrepresentation of the origin of that material, or
+    requiring that modified versions of such material be marked in
+    reasonable ways as different from the original version; or
+
+    d) Limiting the use for publicity purposes of names of licensors or
+    authors of the material; or
+
+    e) Declining to grant rights under trademark law for use of some
+    trade names, trademarks, or service marks; or
+
+    f) Requiring indemnification of licensors and authors of that
+    material by anyone who conveys the material (or modified versions of
+    it) with contractual assumptions of liability to the recipient, for
+    any liability that these contractual assumptions directly impose on
+    those licensors and authors.
+
+  All other non-permissive additional terms are considered "further
+restrictions" within the meaning of section 10.  If the Program as you
+received it, or any part of it, contains a notice stating that it is
+governed by this License along with a term that is a further
+restriction, you may remove that term.  If a license document contains
+a further restriction but permits relicensing or conveying under this
+License, you may add to a covered work material governed by the terms
+of that license document, provided that the further restriction does
+not survive such relicensing or conveying.
+
+  If you add terms to a covered work in accord with this section, you
+must place, in the relevant source files, a statement of the
+additional terms that apply to those files, or a notice indicating
+where to find the applicable terms.
+
+  Additional terms, permissive or non-permissive, may be stated in the
+form of a separately written license, or stated as exceptions;
+the above requirements apply either way.
+
+  8. Termination.
+
+  You may not propagate or modify a covered work except as expressly
+provided under this License.  Any attempt otherwise to propagate or
+modify it is void, and will automatically terminate your rights under
+this License (including any patent licenses granted under the third
+paragraph of section 11).
+
+  However, if you cease all violation of this License, then your
+license from a particular copyright holder is reinstated (a)
+provisionally, unless and until the copyright holder explicitly and
+finally terminates your license, and (b) permanently, if the copyright
+holder fails to notify you of the violation by some reasonable means
+prior to 60 days after the cessation.
+
+  Moreover, your license from a particular copyright holder is
+reinstated permanently if the copyright holder notifies you of the
+violation by some reasonable means, this is the first time you have
+received notice of violation of this License (for any work) from that
+copyright holder, and you cure the violation prior to 30 days after
+your receipt of the notice.
+
+  Termination of your rights under this section does not terminate the
+licenses of parties who have received copies or rights from you under
+this License.  If your rights have been terminated and not permanently
+reinstated, you do not qualify to receive new licenses for the same
+material under section 10.
+
+  9. Acceptance Not Required for Having Copies.
+
+  You are not required to accept this License in order to receive or
+run a copy of the Program.  Ancillary propagation of a covered work
+occurring solely as a consequence of using peer-to-peer transmission
+to receive a copy likewise does not require acceptance.  However,
+nothing other than this License grants you permission to propagate or
+modify any covered work.  These actions infringe copyright if you do
+not accept this License.  Therefore, by modifying or propagating a
+covered work, you indicate your acceptance of this License to do so.
+
+  10. Automatic Licensing of Downstream Recipients.
+
+  Each time you convey a covered work, the recipient automatically
+receives a license from the original licensors, to run, modify and
+propagate that work, subject to this License.  You are not responsible
+for enforcing compliance by third parties with this License.
+
+  An "entity transaction" is a transaction transferring control of an
+organization, or substantially all assets of one, or subdividing an
+organization, or merging organizations.  If propagation of a covered
+work results from an entity transaction, each party to that
+transaction who receives a copy of the work also receives whatever
+licenses to the work the party's predecessor in interest had or could
+give under the previous paragraph, plus a right to possession of the
+Corresponding Source of the work from the predecessor in interest, if
+the predecessor has it or can get it with reasonable efforts.
+
+  You may not impose any further restrictions on the exercise of the
+rights granted or affirmed under this License.  For example, you may
+not impose a license fee, royalty, or other charge for exercise of
+rights granted under this License, and you may not initiate litigation
+(including a cross-claim or counterclaim in a lawsuit) alleging that
+any patent claim is infringed by making, using, selling, offering for
+sale, or importing the Program or any portion of it.
+
+  11. Patents.
+
+  A "contributor" is a copyright holder who authorizes use under this
+License of the Program or a work on which the Program is based.  The
+work thus licensed is called the contributor's "contributor version".
+
+  A contributor's "essential patent claims" are all patent claims
+owned or controlled by the contributor, whether already acquired or
+hereafter acquired, that would be infringed by some manner, permitted
+by this License, of making, using, or selling its contributor version,
+but do not include claims that would be infringed only as a
+consequence of further modification of the contributor version.  For
+purposes of this definition, "control" includes the right to grant
+patent sublicenses in a manner consistent with the requirements of
+this License.
+
+  Each contributor grants you a non-exclusive, worldwide, royalty-free
+patent license under the contributor's essential patent claims, to
+make, use, sell, offer for sale, import and otherwise run, modify and
+propagate the contents of its contributor version.
+
+  In the following three paragraphs, a "patent license" is any express
+agreement or commitment, however denominated, not to enforce a patent
+(such as an express permission to practice a patent or covenant not to
+sue for patent infringement).  To "grant" such a patent license to a
+party means to make such an agreement or commitment not to enforce a
+patent against the party.
+
+  If you convey a covered work, knowingly relying on a patent license,
+and the Corresponding Source of the work is not available for anyone
+to copy, free of charge and under the terms of this License, through a
+publicly available network server or other readily accessible means,
+then you must either (1) cause the Corresponding Source to be so
+available, or (2) arrange to deprive yourself of the benefit of the
+patent license for this particular work, or (3) arrange, in a manner
+consistent with the requirements of this License, to extend the patent
+license to downstream recipients.  "Knowingly relying" means you have
+actual knowledge that, but for the patent license, your conveying the
+covered work in a country, or your recipient's use of the covered work
+in a country, would infringe one or more identifiable patents in that
+country that you have reason to believe are valid.
+
+  If, pursuant to or in connection with a single transaction or
+arrangement, you convey, or propagate by procuring conveyance of, a
+covered work, and grant a patent license to some of the parties
+receiving the covered work authorizing them to use, propagate, modify
+or convey a specific copy of the covered work, then the patent license
+you grant is automatically extended to all recipients of the covered
+work and works based on it.
+
+  A patent license is "discriminatory" if it does not include within
+the scope of its coverage, prohibits the exercise of, or is
+conditioned on the non-exercise of one or more of the rights that are
+specifically granted under this License.  You may not convey a covered
+work if you are a party to an arrangement with a third party that is
+in the business of distributing software, under which you make payment
+to the third party based on the extent of your activity of conveying
+the work, and under which the third party grants, to any of the
+parties who would receive the covered work from you, a discriminatory
+patent license (a) in connection with copies of the covered work
+conveyed by you (or copies made from those copies), or (b) primarily
+for and in connection with specific products or compilations that
+contain the covered work, unless you entered into that arrangement,
+or that patent license was granted, prior to 28 March 2007.
+
+  Nothing in this License shall be construed as excluding or limiting
+any implied license or other defenses to infringement that may
+otherwise be available to you under applicable patent law.
+
+  12. No Surrender of Others' Freedom.
+
+  If conditions are imposed on you (whether by court order, agreement or
+otherwise) that contradict the conditions of this License, they do not
+excuse you from the conditions of this License.  If you cannot convey a
+covered work so as to satisfy simultaneously your obligations under this
+License and any other pertinent obligations, then as a consequence you may
+not convey it at all.  For example, if you agree to terms that obligate you
+to collect a royalty for further conveying from those to whom you convey
+the Program, the only way you could satisfy both those terms and this
+License would be to refrain entirely from conveying the Program.
+
+  13. Remote Network Interaction; Use with the GNU General Public License.
+
+  Notwithstanding any other provision of this License, if you modify the
+Program, your modified version must prominently offer all users
+interacting with it remotely through a computer network (if your version
+supports such interaction) an opportunity to receive the Corresponding
+Source of your version by providing access to the Corresponding Source
+from a network server at no charge, through some standard or customary
+means of facilitating copying of software.  This Corresponding Source
+shall include the Corresponding Source for any work covered by version 3
+of the GNU General Public License that is incorporated pursuant to the
+following paragraph.
+
+  Notwithstanding any other provision of this License, you have
+permission to link or combine any covered work with a work licensed
+under version 3 of the GNU General Public License into a single
+combined work, and to convey the resulting work.  The terms of this
+License will continue to apply to the part which is the covered work,
+but the work with which it is combined will remain governed by version
+3 of the GNU General Public License.
+
+  14. Revised Versions of this License.
+
+  The Free Software Foundation may publish revised and/or new versions of
+the GNU Affero General Public License from time to time.  Such new versions
+will be similar in spirit to the present version, but may differ in detail to
+address new problems or concerns.
+
+  Each version is given a distinguishing version number.  If the
+Program specifies that a certain numbered version of the GNU Affero General
+Public License "or any later version" applies to it, you have the
+option of following the terms and conditions either of that numbered
+version or of any later version published by the Free Software
+Foundation.  If the Program does not specify a version number of the
+GNU Affero General Public License, you may choose any version ever published
+by the Free Software Foundation.
+
+  If the Program specifies that a proxy can decide which future
+versions of the GNU Affero General Public License can be used, that proxy's
+public statement of acceptance of a version permanently authorizes you
+to choose that version for the Program.
+
+  Later license versions may give you additional or different
+permissions.  However, no additional obligations are imposed on any
+author or copyright holder as a result of your choosing to follow a
+later version.
+
+  15. Disclaimer of Warranty.
+
+  THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
+APPLICABLE LAW.  EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
+HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
+OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
+THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+PURPOSE.  THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
+IS WITH YOU.  SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
+ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
+
+  16. Limitation of Liability.
+
+  IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
+WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
+THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
+GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
+USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
+DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
+PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
+EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
+SUCH DAMAGES.
+
+  17. Interpretation of Sections 15 and 16.
+
+  If the disclaimer of warranty and limitation of liability provided
+above cannot be given local legal effect according to their terms,
+reviewing courts shall apply local law that most closely approximates
+an absolute waiver of all civil liability in connection with the
+Program, unless a warranty or assumption of liability accompanies a
+copy of the Program in return for a fee.
+
+                     END OF TERMS AND CONDITIONS
+
+            How to Apply These Terms to Your New Programs
+
+  If you develop a new program, and you want it to be of the greatest
+possible use to the public, the best way to achieve this is to make it
+free software which everyone can redistribute and change under these terms.
+
+  To do so, attach the following notices to the program.  It is safest
+to attach them to the start of each source file to most effectively
+state the exclusion of warranty; and each file should have at least
+the "copyright" line and a pointer to where the full notice is found.
+
+    Twitter App: A user interface and tools for the Twitter API and Archive.
+    Copyright (C) 2021-22 Harlan J. Iverson
+
+    This program is free software: you can redistribute it and/or modify
+    it under the terms of the GNU Affero General Public License as published by
+    the Free Software Foundation, either version 3 of the License, or
+    (at your option) any later version.
+
+    This program is distributed in the hope that it will be useful,
+    but WITHOUT ANY WARRANTY; without even the implied warranty of
+    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+    GNU Affero General Public License for more details.
+
+    You should have received a copy of the GNU Affero General Public License
+    along with this program.  If not, see <https://www.gnu.org/licenses/>.
+
+Also add information on how to contact you by electronic and paper mail.
+
+  If your software can interact with users remotely through a computer
+network, you should also make sure that it provides a way for users to
+get its source.  For example, if your program is a web application, its
+interface could display a "Source" link that leads users to an archive
+of the code.  There are many ways you could offer source, and different
+solutions will be better for different programs; see section 13 for the
+specific requirements.
+
+  You should also get your employer (if you work as a programmer) or school,
+if any, to sign a "copyright disclaimer" for the program, if necessary.
+For more information on this, and how to apply and follow the GNU AGPL, see
+<https://www.gnu.org/licenses/>.

+ 68 - 0
README.md

@@ -0,0 +1,68 @@
+# Twitter App
+
+The purpose of this app is two fold:
+
+* Import the Twitter Archive into a usable format
+* Provide a User Interface to the Archive and Twitter API
+
+Since the Twitter API only provides 7 days of historical data, 
+we download a copy of the archive and use it to augment live data
+from the API and enhance the user experience with the historical data.
+
+## Status
+
+This is the merger of two projects which are on their way toward meeting in the middle.
+
+Initially the Archive tech was developed for personal reporting, and then my quest into Python brought me
+to web development and I needed a known domain to build for so I could focus on learning the web tech.
+
+The intent of this release is to give a deployable Twitter client away to open source, and then bring the 
+archive functionality into the UI. At the time of release the Archive functionality will be obscure but present
+to who looks for it.
+
+## User Interface
+
+The user interface is built to be as simple to customize as possible for the power user.
+
+It connects to the regular Twitter site using the official method that is well documented 
+on the developer site.
+
+It minimizes the need to use Javascript for customization to the extent possible, 
+making the experience good for programmers who have not mastered modern web browser
+nuances.
+
+## Archive
+
+A Twitter's accounts Archive can be requested from settings; it takes about a day to deliver and is 500MB-2GB,
+depending largely on how much media was posted. An archive of 50k Tweets is about 50MB of raw tex or 20MB compressed,
+and can be requested once per month. The Tweets file is not in a usable format, requiring processing even to
+read into a JSON parser. 
+
+One in memory it's simply a large list of Tweets and requires manual coding to do anything useful. The first thing we do
+is put it into an SQLite3 database for fast and easy querying using  SQL, a proven tool known by non-programmer business analysts worldwide.
+
+Once the archive is in a Database, we need an interface to work with it. Since the data is derived from the live site's data, we fit it into a user interface that works like the regular Twitter app.
+
+## Technology
+
+* Python 3
+* HTMx
+* SQLite3
+
+Using HTMx allows us to get away without much Javascript on the client side. It's a throwback
+to earlier days of web development pre-Angular and Reach and into the era of jQuery Ajax forms,
+but it codifies the operations via custom HTML attributes beginning with `hx-~`.
+
+## Contributing
+
+Since I intend to retain full copyright and re-license portions of the project I am only able to accept contributions with a copyright assignment, similar to how the Apache project works.
+
+I'm open to substantial contributions with that in mind, and more importantly I'd love to pay contributors for their work; as such, you may donate to me and I can share the wealth while maintaining my business constraints. I know this may rub some free software folks the wrong way, such as a younger version of myself. These days one needs to be serious about the business of their software, and so releasing my work to the public using a business unfriendly open source license is the balance I can offer.
+
+## License & Copyright
+
+The project is licensed to the public under the AGPL, requiring disclosure of the source code however it is used.
+
+Please contact me for a commercial or copyleft license for portions of the project.
+
+Copyright (C) 2021-22 Harlan J. Iverson; all rights reserved.

+ 7 - 0
requirements.txt

@@ -0,0 +1,7 @@
+Flask                     ~= 2.1.2
+Flask-Cors                ~= 3.0.10
+Flask-Session             ~= 0.4.0
+json-stream               ~= 1.3.0
+python-dateutil           ~= 2.8.2
+requests                  ~= 2.28.0
+requests-oauthlib         ~= 1.3.1

+ 10 - 0
sample-env.txt

@@ -0,0 +1,10 @@
+FLASK_SECRET=
+
+TWITTER_CLIENT_ID=
+TWITTER_CLIENT_SECRET=
+
+# Path to tweet.js converted into json (barely works, bolting these two things together)
+ARCHIVE_TWEETS_PATH=
+
+# For development on localhost set value to 1
+OAUTHLIB_INSECURE_TRANSPORT=

BIN
static/fake-tweet-activity.png


Những thai đổi đã bị hủy bỏ vì nó quá lớn
+ 0 - 0
static/htmx.js


Những thai đổi đã bị hủy bỏ vì nó quá lớn
+ 1 - 0
static/tachyons.min.css


+ 105 - 0
templates/base.html

@@ -0,0 +1,105 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+	<meta charset="utf-8">
+	<meta name="viewport" content="width=device-width, initial-scale=1">
+	<link rel="stylesheet" href="{{ url_for('.static', filename='tachyons.min.css') }}">
+	<script src="{{ url_for('.static', filename='htmx.js') }}"></script>
+	
+	{% block head %}
+	<title>{{ title | default('No Title') }}</title>
+	{% endblock %}
+<style>
+
+
+.qt-box, .card-box {
+	padding: 4px;
+	border: 1px solid black;
+}
+
+.tweet-actions-box {
+	width: 100%;
+	background-color: lightgrey;
+	padding: 2px;
+	text-align: right;
+}
+
+
+.theme-dark {
+	color: #dfdfdf;
+	background-color: #030303;
+}
+    
+	
+
+.theme-dark a {
+	color: green;
+}
+
+.theme a:visited {
+	color: orange;
+}
+
+.theme-dark .qt-box, .theme-dark .card-box {
+	border-color: #dfdfdf;
+}
+
+.theme-dark .tweet-actions-box {
+	background-color: midnightblue;
+}
+
+.theme-dark textarea, .theme-dark input {
+	background-color: #121212;
+	border-color: #dfdfdf;
+	color: #dfdfdf;
+}
+
+.theme-dark button {
+    background-color: #121212;
+    color: #dfdfdf;
+    border-color: #dfdfdf;
+}
+
+</style>
+</head>
+<body>
+
+<!--
+Flexbox makes logical sense but we'll go with a table-based display
+
+-->
+
+<div class="nav" style="position: fixed; width: 25%">
+<ul>
+	<li><a href="{{ url_for('.get_timeline_home_html') }}">Latest Tweets</a></li>
+	<li><a href="{{ url_for('.get_conversations_html') }}">DMs</a></li>
+	<li><a hx-get="/twitter/data/mentions/{{ twitter_user.id }}?me={{ me }}" hx-target="#tweets" hx-swap="outerHTML">Mentions</a></li>
+	<!--
+	<li><a hx-get="/twitter/data/thread/1592514352690900992?me={{ me }}" hx-target="#tweets" hx-swap="outerHTML">Test Thread</a></li>
+	<li><a hx-get="/twitter/data/conversation/1592596557009801216?me={{ me }}" hx-target="#tweets" hx-swap="outerHTML">Test Conversation</a></li>
+	<li><a hx-get="/twitter/data/tweets?ids=1592637236147027970,1592474342289330176&amp;me={{ me }}" hx-target="#tweets" hx-swap="outerHTML">Test Tweets</a></li>
+	-->
+	<li><a href="{{ url_for('.get_bookmarks_html') }}">Bookmarks</a></li>
+	<li><a href="{{ url_for('.get_logout_html') }}">Logout ({{me}})</a></li>
+
+	
+	<li><a href="javascript:document.body.classList.toggle('theme-dark')">Toggle Dark Mode</a></li>
+	<li><a href="javascript:document.location.reload()">Refresh</a></li>
+</ul>
+
+{% include "partial/compose-form.html" %}
+
+
+{% include "partial/media-upload-form.html" %}
+
+{% include "partial/user-picker.html" %}
+</div>
+
+<div style="width: 75%; left:25%; position: absolute">
+
+{% block content %}{% endblock %}
+
+</div>
+
+</body>
+</html>

+ 14 - 0
templates/conversations.html

@@ -0,0 +1,14 @@
+{% extends "base.html" %}
+
+{% block head %}
+	<title>Messages: {{ user.id }}</title>
+{% endblock %}
+
+
+{% block content %}
+<code><pre>
+
+{{ dm_events | tojson(indent=4) | safe }}
+
+</pre></code>
+{% endblock %}

+ 15 - 0
templates/partial/compose-form.html

@@ -0,0 +1,15 @@
+<form class="compose" hx-post="/twitter/tweets/create?me={{ me }}" hx-swap="outerHTML">
+<h2>Compose</h2>
+<ul>
+	<li><textarea name="text" placeholder="Only the finest..." style="width: 100%" onchange="this.onkeyup()" onkeyup="this.form.querySelector('.compose-length').innerHTML = this.value.length" rows="6" cols="30"></textarea>
+	<li><p class="compose-length w-100" style="text-align: right">0</p>
+	<li><input type="text" name="reply_to_tweet_id" placeholder="Reply to Tweet ID" style="width: 100%">
+	<li><input type="text" name="quote_tweet_id" placeholder="Quote Tweet ID" style="width: 100%">
+	<li><button type="submit" style="margin-top: 0.33em">Post</button>
+</ul>
+</form>
+{% if new_tweet_id %}
+<div class="flash">
+Tweet posted. <a href="/tweet/{{ new_tweet_id }}.html">View</a>.
+</div>
+{% endif %}

+ 7 - 0
templates/partial/media-upload-form.html

@@ -0,0 +1,7 @@
+<form action={{ url_for('.post_media_upload', me=me) }} enctype="multipart/form-data" method="POST">
+<ul>
+	<li><input type="file" name="file1">
+	<li><input type="file" name="file2">
+	<li><button type="submit">Upload Media</button>
+</ul>
+</form>

+ 85 - 0
templates/partial/timeline-tweet.html

@@ -0,0 +1,85 @@
+
+<div class="dtc w-10">
+	<img src="{{ tweet.avi_icon_url }}" alt="Avi">
+</div>
+<div class="dtc w-90 v-top">
+	<p class="w-100 mt0 pt0">
+	<strong><a href="{{ tweet.author_url }}" class="w-100">{{ tweet.display_name }}</a></strong>
+	{% if tweet.author_is_verified %}
+	<small class="verified">[verified]</small>
+	{% endif %}
+	
+	<a href="{{ tweet.author_url }}" class="silver">@{{ tweet.handle }}</a>
+	<a href="{{ tweet.source_url }}">{{ tweet.created_at }}</a>
+	</p>
+	<p class="w-100">{{ tweet.text | replace('<', '&lt;') | replace('\n', '<br>') | safe }}</p>
+
+	{% if tweet.quoted_tweet %}
+	<div class="dt qt-box">
+	<div class="dt-row">
+		{% with tweet = tweet.quoted_tweet %}
+			{% include "partial/timeline-tweet.html" %}
+		{% endwith %}
+
+	</div>
+	</div>
+	{% endif %}
+
+	{% if tweet.photos %}
+	<p class="w-100">
+		<ul>
+		{% for photo in tweet.photos %}
+			<li><img class="w-100" src="{{ photo.preview_image_url }}" crossorigin="" referrerpolicy="no-referrer" onclick="this.src='{{ photo.url }}'"></li>
+		{% endfor %}
+		</ul>
+
+	</p>
+	{% endif %}
+
+	{% if tweet.videos %}
+	<p class="w-100">
+		<p>VIDEOS</p>
+		<ul>
+		{% for video in tweet.videos %}
+			<li><img class="w-100" src="{{ video.preview_image_url }}" crossorigin="" referrerpolicy="no-referrer" onclick="this.src='{{ video.image_url }}'"></li>
+		{% endfor %}
+		</ul>
+
+	</p>
+	{% endif %}
+	
+		{% if tweet.card %}
+		
+
+		<div class="card-box w-100">
+		
+			<p><a href="{{ tweet.card.source_url }}">{{ tweet.card.display_url }}</a></p>
+			<p><strong>{{ tweet.card.title }}</strong></p>
+			<p>{{ tweet.card.content }}</p>
+		</div>
+		{% endif %}
+		
+		{% if tweet.public_metrics %}
+		
+
+		<p class="w-100">replies: {{ tweet.public_metrics.reply_count }},
+						 quotes: {{ tweet.public_metrics.quote_count }},
+						 rt: {{ tweet.public_metrics.retweet_count }},
+						 likes: {{ tweet.public_metrics.like_count }}
+						 </p>
+		{% endif %}
+		
+		{% if tweet.non_public_metrics %}
+		
+
+		
+		<p class="w-100">
+			impressions: {{ tweet.non_public_metrics.impression_count }},
+			clicks: {{ tweet.non_public_metrics.user_profile_clicks }},
+			link clicks: {{ tweet.non_public_metrics.url_link_clicks | default('n/a') }},
+			profile clicks: {{ tweet.non_public_metrics.user_profile_clicks | default('n/a') }}
+		</p>
+		{% endif %}
+	
+	
+</div>

+ 108 - 0
templates/partial/tweets-timeline.html

@@ -0,0 +1,108 @@
+<script>
+	if (!window['dataset']) {
+		window.dataset = {
+			items: [],
+			update: function (items) {
+				dataset.items = dataset.items.concat(items);
+			}
+		}
+	}
+</script>
+<ul id="tweets" class="tweets w-75 center z-0">
+
+{% for tweet in tweets %}
+
+<li class="tweet w-100 dt">
+<script>
+	dataset.update([
+		{{ tweet | tojson }}
+	]);
+
+</script>
+
+
+	{% if tweet.retweeted_by %}
+	<div class="dt-row moon-gray">
+		<p class="dtc w-10 tr pa1">RT</p>
+		<p class="dtc w-90"><a class="moon-gray" href="{{ tweet.retweeted_by_url }}">{{ tweet.retweeted_by }} Retweeted</a></p>
+	</div>
+	{% endif %}
+	<div class="dt-row">
+		{% include "partial/timeline-tweet.html" %}
+		
+				
+
+		
+		
+	</div>
+	<div class="dt-row">
+		<div class="dtc"></div>
+		<div class="dtc ">
+		
+		
+		<p class="tweet-actions-box">
+		
+		<a hx-get="/twitter/data/thread/{{ tweet.conversation_id }}?me={{ me }}" hx-target="#tweets" hx-swap="outerHTML">view author thread</a>
+		|
+		<a hx-get="/twitter/data/conversation/{{ tweet.conversation_id }}?me={{ me }}" hx-target="#tweets" hx-swap="outerHTML">view full convo</a>
+		|
+		<a class="tweet-action bookmark" href="#">bookmark</a>
+		<a class="tweet-action copy-formatted" href="#">copy formatted</a>
+		|
+		<a class="tweet-action swipe-to-note" href="#">swipe to note</a>
+		</p>
+		</div>
+	</div>
+	
+
+	</li>
+
+{% endfor %}
+
+
+
+{% if query.next_data_url %}
+
+	<li style="height: 50px; vertical-align: middle"
+		hx-get="{{ query.next_data_url }}"
+		hx-trigger="revealed"
+		hx-swap="outerHTML"
+		hx-select="ul#tweets > li"
+		>
+		<center style="height: 100%">
+
+		<span class="js-only">
+		Loading more tweets...
+		</span>
+		
+		{% if query.next_page_url %}
+		<a href="{{ query.next_page_url }}">
+		Go to Next Page
+		</a>
+		{% endif %}
+		
+		<script>
+			var profileDataEl = document.querySelector('#profile-data');
+			
+			if (window['dataset'] && profileDataEl) {
+				profileDataEl.innerHTML = dataset.items.filter(i => 'public_metrics' in i).map(i => i.public_metrics.like_count).join(', ');
+			}
+
+
+		</script>
+		</center>
+	</li>
+
+{% endif %}
+
+</ul>
+		
+<script>
+	var profileDataEl = document.querySelector('#profile-data');
+	
+	if (window['dataset'] && profileDataEl) {
+		profileDataEl.innerHTML = dataset.items.filter(i => 'public_metrics' in i).map(i => i.public_metrics.like_count).join(', ');
+	}
+
+
+</script>

+ 10 - 0
templates/partial/user-picker.html

@@ -0,0 +1,10 @@
+<ul>
+	{% for k, v in session.items() %}
+		{% if k.startswith('twitter:') %}
+			<li><a href="{{ url_for('twitter_v2_facade.get_profile_html', me=k, user_id=v.id) }}">{{ k }}</a>
+		{% endif %}
+	{% endfor %}
+	{% if archive_enabled %}
+		<li><a href="{{ url_for('twitter_archive_facade.get_profile_html', user_id=0) }}">Archive</a>
+	{% endif %}
+</ul>

+ 12 - 0
templates/tweet-collection.html

@@ -0,0 +1,12 @@
+{% extends "base.html" %}
+
+{% block head %}
+	<title>Tweet Library: {{ user.id }}</title>
+{% endblock %}
+
+
+{% block content %}
+
+	{% include "partial/tweets-timeline.html" %}
+
+{% endblock %}

+ 30 - 0
templates/user-profile.html

@@ -0,0 +1,30 @@
+{% extends "base.html" %}
+
+{% block head %}
+	<title>Profile: {{ user.id }}</title>
+{% endblock %}
+
+
+{% block content %}
+
+	<div id="profile-data" class="w-100" style="position: fixed; height: 30px; background-color: silver">
+	
+	Data
+	
+	</div>
+	<div id="profile-data" style="position: fixed; top: 30px; background-color: silver; width: 80%">
+		<center>
+		<img src="{{ url_for('.static', filename='fake-tweet-activity.png') }}" alt="fake tweet activity" style="width: 60%; height: 40px;">
+		</center>
+	</div>
+	fake-tweet-activity.png
+
+	<div class="w-100" style="margin-top: 80px">
+		<div class="w-100" style="height: 40px">
+			<a href="{{ url_for('.get_profile_html', user_id=user.id, exclude_replies = 1) }}">Without replies</a>
+			|
+			<a href="{{ url_for('.get_profile_html', user_id=user.id, exclude_replies = 0) }}">With replies</a>
+		</div>
+		{% include "partial/tweets-timeline.html" %}
+	</div>
+{% endblock %}

+ 510 - 0
tweet_source.py

@@ -0,0 +1,510 @@
+import json
+import requests
+import sqlite3
+
+class ArchiveTweetSource:
+    """
+    id, created_at, retweeted, favorited, retweet_count, favorite_count, full_text, in_reply_to_status_id_str, in_reply_to_user_id, in_reply_to_screen_nam
+    """
+    def __init__ (self, archive_path, db_path = "data/tweet.db", archive_user_id = None):
+        self.archive_path = archive_path
+        self.user_id = archive_user_id
+        self.db_path = db_path
+        return
+    
+    def get_db (self):
+        db = sqlite3.connect(self.db_path)
+        
+        return db
+
+    def get_user_timeline (self,
+                         author_id = None, max_results = 10, since_id = None):
+    
+        if max_results == None:
+            max_results = -1
+            
+        
+        sql_params = []
+        where_sql = []
+        
+        # if the ID is not stored as a number (eg. string) then this could be a problem
+        if since_id:
+            where_sql.append("id > ?")
+            sql_params.append(since_id)
+            
+        #if author_id:
+        #    where_sql.append("author_id = ?")
+        #    sql_params.append(author_id)
+        
+        where_sql = " and ".join(where_sql)
+        
+        sql_cols = "id, created_at, retweeted, favorited, retweet_count, favorite_count, full_text, in_reply_to_status_id_str, in_reply_to_user_id, in_reply_to_screen_name"
+        
+        if author_id:
+            sql_cols += ", '{}' as author_id".format(author_id)
+        
+        if where_sql:
+            where_sql = "where {}".format(where_sql)
+        
+        sql = "select {} from tweet {} order by created_at asc limit ?".format(sql_cols, where_sql)
+        sql_params.append(max_results)
+        
+        
+        db = self.get_db()
+        
+        cur = db.cursor()
+        cur.row_factory = sqlite3.Row
+        
+        print(sql)
+        print(sql_params)
+        
+        results = list(map(dict, cur.execute(sql, sql_params).fetchall()))
+        
+        return results
+    
+    def get_tweet (self, id_):
+        return self.get_tweets([id_])
+    
+    def get_tweets (self,
+                    ids):
+                    
+        sql_params = []
+        where_sql = []
+        if since_id:
+            ids_in_list_sql = "id in ({})".format( ','.join(['?'] * len(ids)))
+            where_sql.append(ids_in_list_sql)
+            sql_params += ids
+        
+        where_sql = " and ".join(where_sql)
+        
+        sql = "select * from tweet where {} limit ?".format(where_sql)
+        
+        db = self.get_db()
+        
+        cur = db.cursor()
+        cur.row_factory = sqlite3.Row
+        
+        results = list(map(dict, cur.execute(sql, sql_params).fetchall()))
+        
+        return results
+    
+    def search_tweets (self,
+                       query,
+                       since_id = None,
+                       max_results = 10,
+                       sort_order = None
+                       ):
+        
+        return
+
+# https://developer.twitter.com/en/docs/twitter-api/v1/tweets/curate-a-collection/api-reference/get-collections-entries
+# we can perhaps steal a token from the TweetDeck Console, otherwise we need to apply for Standard v1.1 / Elevated
+class ApiV11TweetCollectionSource:
+    def __init__ (self, token):
+        self.token = token
+        
+    def create_collection (self, name):
+        return
+    
+    def bulk_add_to_collection (self, collection_id, items):
+        return
+    
+    def add_to_collection (self, collection_id, item):
+        return
+        
+    def get_collection_tweets (self, collection_id):
+        return
+
+class TwitterApiV2SocialGraph:
+    def __init__ (self, token):
+        self.token = token
+        
+    def get_user (user_id, is_username=False):
+        # GET /2/users/:id
+        # GET /2/users/by/:username
+        return
+    
+    def get_users (user_ids, are_usernames=False):
+        # GET /2/users/by?usernames=
+        # GET /2/users?ids=
+        return
+        
+    def get_following (user_id, 
+                        max_results = 10, pagination_token = None):
+        # GET /2/users/:id/following
+        return
+        
+    def get_followers (user_id,
+                        max_results = 10, pagination_token = None):
+        # GET /2/users/:id/followers
+        return
+        
+    def follow_user (user_id, target_user_id):
+        # POST /2/users/:id/following
+        # {target_user_id}
+        return
+        
+    def unfollow_user (user_id, target_user_id):
+        # DELETE /2/users/:source_user_id/following/:target_user_id
+        return
+
+class ApiV2TweetSource:
+    def __init__ (self, token):
+        self.token = token
+        
+        
+    def create_tweet (self, text, 
+        reply_to_tweet_id = None, quote_tweet_id = None):
+        
+        url = "https://api.twitter.com/2/tweets"
+        
+        tweet = {
+            'text': text
+        }
+        
+        if reply_to_tweet_id:
+            tweet['reply'] = {
+                'in_reply_to_tweet_id': reply_to_tweet_id
+            }
+            
+        if quote_tweet_id:
+            tweet['quote_tweet_id'] = quote_tweet_id
+        
+        body = json.dumps(tweet)
+        
+        headers = {
+            'Authorization': 'Bearer {}'.format(self.token),
+            'Content-Type': 'application/json'
+        }
+        
+        response = requests.post(url, data=body, headers=headers)
+        result = json.loads(response.text)
+        
+        return result
+    
+    def retweet_tweet( self, user_id, tweet_id ):
+        
+        url = "https://api.twitter.com/2/users/{}/retweets".format(user_id)
+        
+        retweet = {
+            'tweet_id': tweet_id
+        }
+        
+        body = json.dumps(retweet)
+        
+        headers = {
+            'Authorization': 'Bearer {}'.format(self.token),
+            'Content-Type': 'application/json'
+        }
+        
+        response = requests.post(url, data=body, headers=headers)
+        result = json.loads(response.text)
+        
+        return result
+        
+    
+    def get_home_timeline (self, user_id, variant = 'reverse_chronological', max_results = 10, pagination_token = None, since_id = None):
+        """
+        Get a user's timeline as viewed by the user themselves.
+        """
+        
+        path = 'users/{}/timelines/{}'.format(user_id, variant)
+        
+        return self.get_timeline(path, 
+            max_results=max_results, pagination_token=pagination_token, since_id=since_id) 
+    
+    def get_timeline (self, path,
+        max_results = 10, pagination_token = None, since_id = None,
+        non_public_metrics = False,
+        exclude_replies=False):
+        """
+        Get any timeline, including custom curated timelines built by Tweet Deck / ApiV11.
+        """
+        
+        token = self.token
+        
+        url = "https://api.twitter.com/2/{}".format(path)
+        
+        tweet_fields = ["created_at", "conversation_id",  "referenced_tweets", "text", "public_metrics", "entities", "attachments"]
+        media_fields = ["alt_text", "type", "preview_image_url", "public_metrics", "url", "media_key", "duration_ms", "width", "height", "variants"]
+        user_fields = ["created_at", "name", "username", "location", "profile_image_url", "verified"]
+        expansions = ["entities.mentions.username",
+                    "attachments.media_keys",
+                    "author_id",
+                    "referenced_tweets.id",
+                    "referenced_tweets.id.author_id"]
+        
+        if non_public_metrics:
+            tweet_fields.append("non_public_metrics")
+            media_fields.append("non_public_metrics")
+        
+        params = {
+            "expansions": ",".join(expansions),
+            "media.fields": ",".join(media_fields),
+            
+            "tweet.fields": ",".join(tweet_fields),
+            "user.fields": ",".join(user_fields),
+            
+            "max_results": max_results,
+        }
+        
+        exclude = []
+        
+        if exclude_replies:
+            exclude.append('replies')
+            
+        if len(exclude):
+            params['exclude'] = exclude
+        
+        
+        if pagination_token:
+            params['pagination_token'] = pagination_token
+            
+        if since_id:
+            params['since_id'] = since_id
+        
+        headers = {"Authorization": "Bearer {}".format(token)}
+        
+        #headers = {"Authorization": "access_token {}".format(access_token)}
+
+        response = requests.get(url, params=params, headers=headers)
+        response_json = json.loads(response.text)
+        
+        return response_json
+    
+    def get_mentions_timeline (self, user_id,
+                                max_results = 10, pagination_token = None, since_id = None):
+                                
+        path = "users/{}/mentions".format(user_id)
+        
+        return self.get_timeline(path, 
+            max_results=max_results, pagination_token=pagination_token, since_id=since_id)
+    
+    def get_user_timeline (self, user_id,
+                          max_results = 10, pagination_token = None, since_id = None,
+                          non_public_metrics=False,
+                          exclude_replies=False):
+        """
+        Get a user's Tweets as viewed by another.
+        """
+        path = "users/{}/tweets".format(user_id)
+        
+        return self.get_timeline(path, 
+            max_results=max_results, pagination_token=pagination_token, since_id=since_id,
+            non_public_metrics = non_public_metrics,
+            exclude_replies=exclude_replies)
+    
+    
+    def get_tweet (self, id_, non_public_metrics = False):
+        return self.get_tweets([id_], non_public_metrics = non_public_metrics)
+    
+    def get_tweets (self,
+                    ids,
+                    non_public_metrics = False):
+                    
+        token = self.token
+        
+        url = "https://api.twitter.com/2/tweets"
+        
+        tweet_fields = ["created_at", "conversation_id",  "referenced_tweets", "text", "public_metrics", "entities", "attachments"]
+        media_fields = ["alt_text", "type", "preview_image_url", "public_metrics", "url", "media_key", "duration_ms", "width", "height", "variants"]
+        user_fields = ["created_at", "name", "username", "location", "profile_image_url", "verified"]
+        expansions = ["entities.mentions.username",
+                    "attachments.media_keys",
+                    "author_id",
+                    "referenced_tweets.id",
+                    "referenced_tweets.id.author_id"]
+        
+        if non_public_metrics:
+            tweet_fields.append("non_public_metrics")
+            media_fields.append("non_public_metrics")
+
+        params = {
+            "ids": ','.join(ids),
+            "expansions": ",".join(expansions),
+            "media.fields": ",".join(media_fields),
+            
+            "tweet.fields": ",".join(tweet_fields),
+            "user.fields": ",".join(user_fields)
+        }
+        headers = {"Authorization": "Bearer {}".format(token)}
+        
+        response = requests.get(url, params=params, headers=headers)
+        response_json = json.loads(response.text)
+        
+        return response_json
+    
+    def search_tweets (self,
+                       query, 
+                       pagination_token = None,
+                       since_id = None,
+                       max_results = 10,
+                       sort_order = None,
+                       non_public_metrics = False
+                       ):
+        
+        token = self.token
+        
+        url = "https://api.twitter.com/2/tweets/search/recent"
+        
+        tweet_fields = ["created_at", "conversation_id",  "referenced_tweets", "text", "public_metrics", "entities", "attachments"]
+        media_fields = ["alt_text", "type", "preview_image_url", "public_metrics", "url", "media_key", "duration_ms", "width", "height", "variants"]
+        user_fields = ["created_at", "name", "username", "location", "profile_image_url", "verified"]
+        expansions = ["entities.mentions.username",
+                    "attachments.media_keys",
+                    "author_id",
+                    "referenced_tweets.id",
+                    "referenced_tweets.id.author_id"]
+        
+        if non_public_metrics:
+            tweet_fields.append("non_public_metrics")
+            media_fields.append("non_public_metrics")
+        
+        params = {
+            "expansions": ",".join(expansions),
+            "media.fields": ",".join(media_fields),
+            
+            "tweet.fields": ",".join(tweet_fields),
+            "user.fields": ",".join(user_fields),
+            
+            "query": query,
+            "max_results": max_results,
+        }
+        
+        if pagination_token:
+            params['pagination_token'] = pagination_token
+            
+        if since_id:
+            params['since_id'] = since_id
+        
+        if sort_order:
+            params['sort_order'] = sort_order
+        
+        
+        headers = {"Authorization": "Bearer {}".format(token)}
+        
+        response = requests.get(url, params=params, headers=headers)
+        response_json = json.loads(response.text)
+        
+        return response_json
+        
+        
+    
+    def count_tweets (self,
+                       query, 
+                       since_id = None,
+                       granularity = 'hour'
+                       ):
+        
+        token = self.token
+        
+        url = "https://api.twitter.com/2/tweets/counts/recent"
+        
+        
+        params = {
+            "query": query
+        }
+
+        if since_id:
+            params['since_id'] = since_id
+        
+        headers = {"Authorization": "Bearer {}".format(token)}
+        
+        response = requests.get(url, params=params, headers=headers)
+        
+        print(response.status_code)
+        print(response.text)
+        
+        response_json = json.loads(response.text)
+        
+        return response_json
+
+    #def get_conversation (self, tweet_id, pagination_token = None,
+    # TODO
+
+    def get_thread (self, tweet_id,
+                       author_id = None,
+                       pagination_token = None,
+                       since_id = None,
+                       max_results = 10,
+                       sort_order = None
+                       ):
+        
+        # FIXME author_id can be determined from a Tweet object
+        query = "conversation_id:{}".format(tweet_id)
+        if author_id:
+            query += " from:{}".format(author_id)
+            
+        return self.search_tweets(query, 
+            pagination_token = pagination_token, since_id = since_id, max_results = max_results, sort_order = sort_order)
+    
+    def get_bookmarks (self, user_id,
+                          max_results = 10, pagination_token = None, since_id = None):
+        path = "users/{}/bookmarks".format(user_id)
+        
+        return self.get_timeline(path, 
+            max_results=max_results, pagination_token=pagination_token, since_id=since_id)
+    
+    def get_media_tweets (self,
+                   author_id = None,
+                   has_media = True,
+                   has_links = None,
+                   has_images = None,
+                   has_videos = None,
+                   pagination_token = None,
+                   since_id = None,
+                   max_results = 10,
+                   sort_order = None
+                   ):
+        
+        # FIXME author_id can be determined from a Tweet object
+        
+        query = ""
+        if has_media != None:
+            if not has_media:
+                query += "-"
+            query += "has:media "
+                
+        if has_links != None:
+            if not has_links:
+                query += " -"
+            query += "has:links "
+            
+        if has_images != None:
+            if not has_images:
+                query += " -"
+            query += "has:images "
+            
+        if has_videos != None:
+            if not has_videos:
+                query += " -"
+            query += "has:videos "
+            
+        
+        if author_id:
+            query += "from:{} ".format(author_id)
+            
+        return self.search_tweets(query, 
+            pagination_token = pagination_token, since_id = since_id, max_results = max_results, sort_order = sort_order)
+    
+        
+    def get_retweets (self, tweet_id):
+        # GET /2/tweets/:id/retweeted_by
+        return 
+        
+    def get_quote_tweets( self, tweet_id):
+        # GET /2/tweets/:id/quote_tweets
+        return 
+        
+    def get_likes (self, tweet_id):
+        # GET /2/tweets/:id/liking_users
+        return 
+        
+    def get_liked_by (self, user_id):
+        #  GET /2/users/:id/liked_tweets
+        return 
+    
+    def get_list_tweets (self, list_id):
+        # GET /2/lists/:id/tweets
+        return
+    

+ 62 - 0
twitter_app.py

@@ -0,0 +1,62 @@
+import os
+from configparser import ConfigParser
+
+from flask import Flask, g, redirect, url_for
+from flask_cors import CORS
+
+from twitter_v2_facade import twitter_app as twitter_v2
+from twitter_archive_facade import twitter_app as twitter_archive
+
+
+def import_env ():
+    cp = ConfigParser()
+    if os.path.exists('.env'):
+        with open('.env') as stream:
+            cp.read_string('[default]\n' + stream.read())
+            os.environ.update(dict(cp['default']))
+
+if __name__ == '__main__':
+    import_env()
+
+    PORT = int(os.environ.get('PORT', 5000))
+    HOST = os.environ.get('HOST', '127.0.0.1')
+    
+    archive_enabled = os.environ.get('ARCHIVE_TWEETS_PATH') and True
+    glitch_enabled = os.environ.get('PROJECT_DOMAIN') and True
+    
+    api = Flask(__name__, static_url_path='')
+    
+    
+
+    @api.before_request
+    def add_config ():
+        g.glitch_enabled = glitch_enabled
+        
+        if glitch_enabled:
+            g.app_url = 'https://{}.glitch.me'.format( os.environ.get('PROJECT_DOMAIN') )
+        else:
+            g.app_url = 'http://{}:{}'.format('localhost', PORT)
+        
+
+    
+    @api.context_processor
+    def inject_config ():
+        return {'archive_enabled': archive_enabled}
+    
+    api.secret_key = os.environ.get('FLASK_SECRET')
+    
+    api.config['TEMPLATES_AUTO_RELOAD'] = True
+
+    api.register_blueprint(twitter_v2, url_prefix='/twitter')
+    
+    if archive_enabled:
+        api.register_blueprint(twitter_archive, url_prefix='/twitter-archive')
+    
+    CORS(api)
+    
+    @api.route('/')
+    def index ():
+        return redirect(url_for('.twitter_v2_facade.get_login_html'))
+    
+    
+    api.run(port=PORT, host=HOST)

+ 522 - 0
twitter_archive_facade.py

@@ -0,0 +1,522 @@
+from configparser import ConfigParser
+import base64
+from flask import Flask, json, Response, render_template, request, send_from_directory, Blueprint, url_for, g
+from flask_cors import CORS
+import sqlite3
+import os
+import json
+import json_stream
+
+from zipfile import ZipFile
+import itertools
+
+import datetime
+import dateutil
+import dateutil.parser
+import dateutil.tz
+
+import requests
+
+
+from tweet_source import ArchiveTweetSource
+
+ARCHIVE_TWEETS_PATH=os.environ.get('ARCHIVE_TWEETS_PATH', 'data/tweets.json')
+
+
+twitter_app = Blueprint('twitter_archive_facade', 'twitter_archive_facade',
+    static_folder='static',
+    static_url_path='',
+    url_prefix='/')
+
+
+
+@twitter_app.before_request
+def add_me ():
+    #if me.startswith('twitter') and me in session:
+    g.twitter_user = {'id': '0'}
+    
+
+
+@twitter_app.context_processor
+def inject_me():
+    
+    return {'twitter_user': g.twitter_user}
+    
+    
+    
+# ---------------------------------------------------------------------------------------------------------
+# ---------------------------------------------------------------------------------------------------------
+#     Tweet Archive and old tests
+# ---------------------------------------------------------------------------------------------------------
+# ---------------------------------------------------------------------------------------------------------
+
+
+
+# https://stackoverflow.com/questions/48218065/programmingerror-sqlite-objects-created-in-a-thread-can-only-be-used-in-that-sa
+db = sqlite3.connect(":memory:", check_same_thread=False)
+db_need_init = True
+
+if db_need_init:
+  print("Creating tweet db...")
+  db.execute("create table tweet (id, created_at, content)")
+  
+  
+
+def tweets_js_to_json (path, to_path):
+  # open JS file provided in archive and convert it to JSON
+  # string manipulation should be enough
+  return True
+  
+  
+def populate_tweetsdb_from_compressed_json (db, tweets_json_path):
+
+    # perf: we should find a batch size for executemany if this is too slow.
+    # https://stackoverflow.com/questions/43785569/for-loop-or-executemany-python-and-sqlite3
+
+    ti = open(tweets_json_path)
+    data = json_stream.load(ti)
+    for tweet in data.persistent():
+        reply = None
+        if "reply" in tweet:
+            reply = tweet["reply"]
+        values = [tweet["id"], tweet["full_text_length"], tweet["date"], reply]
+        db.execute("insert into tweet (id, full_text_length, date, reply) values (?, ?, ?, ?)", values)
+    ti.close()
+
+    return True
+
+def print_retweets (tweets_path):
+    tweets_file = open(tweets_path, 'rt', encoding='utf-8')
+    tweets_data = json_stream.load(tweets_file)
+    print('[')
+    for t in tweets_data:
+       tweet = t.persistent()['tweet']
+       if int(tweet['retweet_count']) > 1:
+           print(json.dumps({'id': tweet['id'], 'x': tweet['created_at'], 'y': tweet['retweet_count']}) + ',')
+    print(']')
+    tweets_file.close()
+    
+    return True
+
+def tweet_to_actpub (t):
+
+  return t
+
+@twitter_app.route('/tweets/isd', methods=['GET'])
+def get_tweets_isd ():
+  # simulate GraphQL conventions with REST:
+  # created_at[gte]=
+  # created_at[lte]=
+  # author=
+  # content[re]=
+  # expansions=media,...
+  #results = langs_con.execute("select rowid, id, created_at, content from tweet").fetchall()
+  #return Response(json.dumps(results), mimetype='application/json')
+  return send_from_directory('data', 'tweets-ispoogedaily.json')
+
+@twitter_app.route('/tweets/storms', methods=['GET'])
+def get_tweet_storms ():
+  #content = open('data/storm-summaries-2021.json').read()
+  #return Response(content, mimetype='application/json')
+  return send_from_directory('data', 'storm-summaries-2021.json')
+
+
+
+@twitter_app.route('/bookmarks', methods=['GET'])
+def get_bookmarks ():
+  #content = open('data/storm-summaries-2021.json').read()
+  #return Response(content, mimetype='application/json')
+  return send_from_directory('data', 'bookmarks-ispoogedaily.json')
+
+
+@twitter_app.route('/timeline', methods=['GET'])
+def get_timeline ():
+  #content = open('data/storm-summaries-2021.json').read()
+  #return Response(content, mimetype='application/json')
+  return send_from_directory('data', 'timeline-minimal.json')
+
+
+
+
+@twitter_app.route('/tweets/compressed', methods=['POST'])
+def post_tweets_compressed ():
+  db_exists = os.path.exists("tweets.db")
+  
+  
+  if not db_exists:
+    db = sqlite3.connect("tweets.db")
+    db.execute("create table tweet (id, full_text_length, date, reply)")
+    populate_tweetsdb_from_compressed_json(db, "data/tweet-items.json")
+    db.commit()
+    db.close()
+  
+  #content = open('data/storm-summaries-2021.json').read()
+  #return Response(content, mimetype='application/json')
+  return Response("ok")
+
+
+tweets_form_meta_data = {
+    'fields': [
+        {'name': 'id'},
+        {'name': 'created_at', 'type': 'date'},
+        {'name': 'retweeted', 'type': 'boolean'},
+        {'name': 'favorited', 'type': 'boolean'},
+        {'name': 'retweet_count', 'type': 'int'},
+        {'name': 'favorite_count', 'type': 'int'},
+        {'name': 'full_text', 'type': 'string', 'searchable': True},
+        {'name': 'in_reply_to_status_id_str', 'type': 'string'},
+        {'name': 'in_reply_to_user_id', 'type': 'string'},
+        {'name': 'in_reply_to_screen_name', 'type': 'string'}
+        ],
+    'id': 'id',
+    'root': 'tweets',
+    'url': '/tweets/search',
+    'access': ['read']
+}
+
+@twitter_app.route('/tweets/form', methods=['GET'])
+def get_tweets_form ():
+    
+    response_body = {
+        'metaData': tweets_form_meta_data
+        }
+
+    return Response(json.dumps(response_body), mimetype="application/json")
+    
+   
+def db_tweet_to_card (tweet):
+
+    user = {'username': 'ispoogedaily', 'id': '14520320'}
+
+    tweet_url = 'https://twitter.com/{}/status/{}'.format(user['username'], tweet['id'])
+    
+    content = tweet['full_text'] + "\n\n[view tweet]({})".format(tweet_url)
+    
+    card = {
+      'id': 'tweet-' + tweet['id'],
+      'content': content,
+      'content_type': 'text/plain',
+      'created_at': tweet['created_at'],
+      'modified_at': None,
+      'title': '@' + user['username'] + ' at ' + tweet['created_at'],
+      'content_source': tweet_url,
+      #'tweet': tweet,
+      #'user': user
+    }
+    
+    return card
+    
+   
+# tweetStore = new Ext.data.JsonStore({'url': 'http://localhost:5004/tweets/search.rows.json', 'autoLoad': true})
+
+
+
+def tweet_model (tweet_data):
+    # retweeted_by, avi_icon_url, display_name, handle, created_at, text
+    
+    """
+    {"id": "797839193", "created_at": "2008-04-27T04:00:27", "retweeted": 0, "favorited": 0, "retweet_count": "0", "favorite_count": "0", "full_text": "Putting pizza on. Come over any time!", "in_reply_to_status_id_str": null, "in_reply_to_user_id": null, "in_reply_to_screen_name": null, "author_id": "14520320"}, {"id": "797849979", "created_at": "2008-04-27T04:27:46", "retweeted": 0, "favorited": 0, "retweet_count": "0", "favorite_count": "0", "full_text": "hijacked!@!!!", "in_reply_to_status_id_str": null, "in_reply_to_user_id": null, "in_reply_to_screen_name": null, "author_id": "14520320"}
+    """
+    t = {
+        'id': tweet_data['id'],
+        'text': tweet_data['full_text'],
+        'created_at': tweet_data['created_at'],
+        'author_is_verified': False,
+        
+        'conversation_id': tweet_data['id'],
+        
+        'avi_icon_url': '',
+        
+        'display_name': 'Archive User',
+        'handle': '!archive',
+        
+        'author_url': url_for('.get_profile_html', user_id='0'),
+        'author_id': '0',
+        
+        'source_url': '!source_url',
+        'source_author_url': '!source_author_url',
+        #'is_edited': len(tweet_data['edit_history_tweet_ids']) > 1
+        
+        
+    }
+
+    
+    t['public_metrics'] = {
+        'like_count': tweet_data['favorite_count'],
+        'retweet_count': tweet_data['retweet_count'],
+        'reply_count': 0,
+        'quote_count': 0
+    }
+    
+    return t
+
+
+@twitter_app.route('/data/timeline/user/<user_id>')
+def get_data_timeline_user (user_id):
+
+    pagination_token = request.args.get('pagination_token') # since_id
+    #exclude_replies = request.args.get('exclude_replies')
+    
+    #is_me = user_id == twitter['id']
+    
+    tweet_source = ArchiveTweetSource(ARCHIVE_TWEETS_PATH)
+    
+    db_tweets = tweet_source.get_user_timeline(author_id = user_id,
+                                                    since_id = pagination_token,
+                                                    #exclude_replies = exclude_replies == '1'
+                                                    )
+                                                    
+
+    tweets = list(map(tweet_model, db_tweets))
+    next_token = db_tweets[-1]['id']
+    
+    query = {}
+    
+    if next_token:
+        query = {
+            **query,
+            
+            'next_data_url': url_for('.get_data_timeline_user', user_id=user_id, pagination_token=next_token)
+        }
+    
+    if 'HX-Request' in request.headers:
+        user = {
+            'id': user_id
+        }
+        
+        return render_template('partial/tweets-timeline.html', user = user, tweets = tweets, query = query)
+    else:
+        response_body = json.dumps({
+            'tweets': tweets,
+            'query': query
+        })
+        return Response(response_body, mimetype='application/json')
+
+
+
+@twitter_app.route('/profile/<user_id>.html', methods=['GET'])
+def get_profile_html (user_id):
+
+    pagination_token = request.args.get('pagination_token')
+    #exclude_replies = request.args.get('exclude_replies', '1')
+
+    tweet_source = ArchiveTweetSource(ARCHIVE_TWEETS_PATH)
+    
+    db_tweets = tweet_source.get_user_timeline(author_id = user_id,
+                                                    since_id = pagination_token,
+                                                    #exclude_replies = exclude_replies == '1'
+                                                    )
+                                                    
+
+    tweets = list(map(tweet_model, db_tweets))
+    next_token = db_tweets[-1]['id']
+
+
+    query = {}
+    
+    if next_token:
+        query = {
+            **query,
+            
+            'next_data_url': url_for('.get_data_timeline_user', user_id=user_id, pagination_token=next_token, exclude_replies=1),
+            'next_page_url': url_for('.get_profile_html', user_id=user_id , pagination_token=next_token)
+        }
+    
+    profile_user = {
+            'id': user_id
+        }
+    
+    return render_template('user-profile.html', user = profile_user, tweets = tweets, query = query)
+
+
+@twitter_app.route('/latest.html', methods=['GET'])
+def get_timeline_home_html (variant = "reverse_chronological", pagination_token=None):
+    return 'ok'
+    
+    
+@twitter_app.route('/conversations.html', methods=['GET'])
+def get_conversations_html ():
+    return 'ok'
+
+@twitter_app.route('/bookmarks.html', methods=['GET'])
+def get_bookmarks_html (user_id):
+    return 'ok'
+
+@twitter_app.route('/logout.html', methods=['GET'])
+def get_logout_html ():
+    return 'ok'
+    
+@twitter_app.route('/media/upload', methods=['POST'])
+def post_media_upload ():
+    return 'ok'
+
+
+@twitter_app.route('/tweets/search', methods=['GET'])
+@twitter_app.route('/tweets/search.<string:response_format>', methods=['GET'])
+def get_tweets_search (response_format='json'):
+    
+    search = request.args.get('q')
+    limit = int(request.args.get('limit', 10))
+    offset = int(request.args.get('offset', 0))
+    
+    in_reply_to_user_id = int(request.args.get('in_reply_to_user_id', 0))
+    
+    db = sqlite3.connect('data/tweet.db')
+    
+    sql = """
+select
+id, created_at, retweeted, favorited, retweet_count, favorite_count, full_text, in_reply_to_status_id_str, in_reply_to_user_id, in_reply_to_screen_name
+from tweet
+    """
+    
+    sql_params = []
+    
+    if search:
+        sql += " where full_text like ?"
+        sql_params.append("%{}%".format(search))
+        
+    if in_reply_to_user_id:
+        sql += " where in_reply_to_user_id = ?"
+        sql_params.append(str(in_reply_to_user_id))
+        
+    if limit:
+        sql += ' limit ?'
+        sql_params.append(limit)
+        
+    if offset:
+        sql += ' offset ?'
+        sql_params.append(offset)
+    
+    cur = db.cursor()
+    cur.row_factory = sqlite3.Row
+    
+    tweets = list(map(dict, cur.execute(sql, sql_params).fetchall()))
+    cur.close()
+    db.close()
+    
+    result = None
+    
+    if response_format  == 'cards.json':
+        cards = list(map(db_tweet_to_card, tweets))
+        
+        result = {
+            "q": search,
+            "cards": cards
+        }
+    elif response_format == 'rows.json':
+        meta = tweets_form_meta_data
+        
+        fields = meta['fields']
+        fields = list(map(lambda f: {**f[1], 'mapping': f[0]}, enumerate(fields)))
+        
+        meta = {**meta, 'fields': fields, 'id': '0'}
+        
+        
+        def tweet_to_row (t):
+            row = list(map(lambda f: t.get(f['name']), fields))
+            
+            return row
+        rows = list(map(tweet_to_row, tweets))
+        result = {
+            "q": search,
+            "metaData": meta,
+            "tweets": rows
+        }
+        
+    else:
+        result = {
+            "q": search,
+            "tweets": tweets
+        }
+    
+    return Response(json.dumps(result), mimetype="application/json")
+
+@twitter_app.route('/tweets', methods=['POST'])
+def post_tweets ():
+    tweets_path = ARCHIVE_TWEETS_PATH
+    
+    tweets_file = open(tweets_path, 'rt', encoding='utf-8')
+    tweets_data = json_stream.load(tweets_file)
+    
+    db = sqlite3.connect('data/tweet.db')
+    
+    db.execute('create table tweet (id, created_at, retweeted, favorited, retweet_count, favorite_count, full_text, in_reply_to_status_id_str, in_reply_to_user_id, in_reply_to_screen_name)')
+    db.commit()
+    
+    i = 0
+    cur = db.cursor()
+    for tweet in tweets_data.persistent():
+        t = dict(tweet['tweet'])
+        
+        dt = dateutil.parser.parse(t['created_at'])
+        dt_utc = dt.astimezone(dateutil.tz.tz.gettz('UTC'))
+        created_at = dt_utc.strftime('%Y-%m-%dT%H:%M:%SZ')
+        
+        sql = 'insert into tweet (id, created_at, retweeted, favorited, retweet_count, favorite_count, full_text, in_reply_to_status_id_str, in_reply_to_user_id, in_reply_to_screen_name) values (?,?,?,?,?,?,?,?,?,?)'
+        
+        tweet_values = [
+            t['id'],
+            created_at,
+            t['retweeted'],
+            t['favorited'],
+            t['retweet_count'],
+            t['favorite_count'],
+            t['full_text'],
+            
+            t.get('in_reply_to_status_id_str'),
+            t.get('in_reply_to_user_id'),
+            t.get('in_reply_to_screen_name')
+        ]
+        
+        cur.execute(sql, tweet_values)
+        
+        i += 1
+        if i % 100 == 0:
+            cur.connection.commit()
+            cur = db.cursor()
+    
+    cur.connection.commit()
+    cur.close()
+    db.close()
+    
+    tweets_file.close()
+
+# ---------------------------------------------------------------------------------------------------------
+# ---------------------------------------------------------------------------------------------------------
+
+
+def tweet_to_card (tweet, includes):
+    
+    user = list(filter(lambda u: u.get('id') == tweet['author_id'], includes.get('users')))[0]
+    
+    tweet_url = 'https://twitter.com/{}/status/{}'.format(user['username'], tweet['id'])
+    
+    content = tweet['text'] + "\n\n[view tweet]({})".format(tweet_url)
+    
+    card = {
+      'id': 'tweet-' + tweet['id'],
+      'content': content,
+      'content_type': 'text/markdown',
+      'created_at': tweet['created_at'], # can be derived from oldest in edit_history_tweet_ids
+      'modified_at': None, # can be derived from newest in edit_history_tweet_ids
+      'title': '@' + user['username'] + ' at ' + tweet['created_at'],
+      'content_source': tweet_url,
+      #'tweet': tweet,
+      #'user': user
+    }
+    
+    return card
+
+
+def response_to_cards (response_json, add_included = True):
+    tweets = response_json.get('data')
+    includes = response_json.get('includes')
+
+    cards = list(map(lambda t: tweet_to_card(t, includes), tweets))
+    
+    if add_included:
+        included_cards = list(map(lambda t: tweet_to_card(t, includes), includes.get('tweets')))
+        cards += included_cards
+    
+    return cards
+

+ 1251 - 0
twitter_v2_facade.py

@@ -0,0 +1,1251 @@
+from configparser import ConfigParser
+import base64
+from flask import json, Response, render_template, request, send_from_directory, Blueprint, session, redirect, g, current_app
+
+from flask_cors import CORS
+import sqlite3
+import os
+import json
+import json_stream
+from zipfile import ZipFile
+import itertools
+
+from io import BufferedReader
+from werkzeug.utils import secure_filename
+
+import datetime
+import dateutil
+import dateutil.parser
+import dateutil.tz
+
+import requests
+
+
+import hashlib
+import re
+from requests.auth import AuthBase, HTTPBasicAuth
+from requests_oauthlib import OAuth2Session
+
+
+from tweet_source import ApiV2TweetSource
+
+
+from flask import url_for as og_url_for
+
+app_access_token = None
+
+app_consumer_key = os.environ.get("TWITTER_CONSUMER_KEY")
+app_secret_key = os.environ.get("TWITTER_CONSUMER_SECRET")
+
+TWITTER_SCOPES = ["bookmark.read", "tweet.read", "tweet.write", "dm.read", "users.read", "offline.access"]
+
+twitter_app = Blueprint('twitter_v2_facade', 'twitter_v2_facade',
+    static_folder='static',
+    static_url_path='',
+    url_prefix='/')
+
+def url_for_with_me (route, *args, **kwargs):
+    #print('url_for_with_me')
+    if route.endswith('.static'):
+        return og_url_for(route, *args, **kwargs)
+    
+    return og_url_for(route, *args, **{'me': g.me, **kwargs})
+
+url_for = url_for_with_me
+
+
+
+@twitter_app.before_request
+def add_me ():
+    g.me = request.args.get('me')
+    
+    #if me.startswith('twitter') and me in session:
+    g.twitter_user = session.get(g.me)
+    
+
+
+@twitter_app.context_processor
+def inject_me():
+    
+    return {'me': g.me, 'twitter_user': g.twitter_user, 'url_for': url_for_with_me}
+    
+
+
+
+
+@twitter_app.route('/logout.html')
+def get_logout_html ():
+    del session[g.me]
+    return redirect('/')
+    
+    
+# def add_me(endpoint, values):
+    ##values['me'] = request.args.get('me')
+    # g.me = request.args.get('me')
+
+# twitter_app.url_value_preprocessor(add_me)
+
+@twitter_app.route('/tokens.html')
+def get_tokens_html ():
+    return url_for('.get_tokens_html', me=g.me)
+    
+@twitter_app.route('/logged-in.html')
+def get_loggedin_html ():
+    client_id = os.environ.get('TWITTER_CLIENT_ID')
+    client_secret = os.environ.get('TWITTER_CLIENT_SECRET')
+    
+    code_verifier = session['twitter_code_verifier']
+    
+    code = request.args.get('code')
+    state = request.args.get('state')
+    
+    endpoint_url = g.app_url
+    redirect_uri = endpoint_url + og_url_for('.get_loggedin_html')
+    
+    authorization_response = redirect_uri + '?code={}&state={}'.format(code, state)
+    
+    # Fetch your access token
+    token_url = "https://api.twitter.com/2/oauth2/token"
+    
+    
+
+    # The following line of code will only work if you are using a type of App that is a public client
+    auth = False
+
+    # If you are using a confidential client you will need to pass in basic encoding of your client ID and client secret.
+
+    # Please remove the comment on the following line if you are using a type of App that is a confidential client
+    auth = HTTPBasicAuth(client_id, client_secret)
+    
+    scopes = TWITTER_SCOPES
+    #redirect_uri = 'https://{}/api/logged-in'.format(os.environ.get('PROJECT_DOMAIN') + '.glitch.me')
+    oauth = OAuth2Session(client_id, redirect_uri=redirect_uri, scope=scopes)
+    
+
+    token = oauth.fetch_token(
+        token_url=token_url,
+        authorization_response=authorization_response,
+        auth=auth,
+        client_id=client_id,
+        include_client_id=True,
+        code_verifier=code_verifier,
+    )
+
+    # Your access token
+    access = token["access_token"]
+    refresh = token["refresh_token"]
+    expires_at = token["expires_at"] # expires_in
+
+    # Make a request to the users/me endpoint to get your user ID
+    user_me = requests.request(
+        "GET",
+        "https://api.twitter.com/2/users/me",
+        headers={"Authorization": "Bearer {}".format(access)},
+    ).json()
+    user_id = user_me["data"]["id"]
+    user_username = user_me["data"]["username"]
+    user_name = user_me["data"]["name"]
+    
+    del session['twitter_code_verifier']
+    
+    me = 'twitter:{}'.format(user_id)
+    
+    session[ me ] = {
+        'expires_at': expires_at,
+        'access_token': access,
+        'refresh_token': refresh,
+        'id': user_id,
+        'display_name': user_name,
+        'username': user_username
+    }
+    
+    g.me = me
+    g.twitter_user = session[ me ]
+    
+    return redirect(url_for('.get_timeline_home_html'))
+  
+@twitter_app.route('/login.html')
+def get_login_html ():
+    client_id = os.environ.get('TWITTER_CLIENT_ID')
+    client_secret = os.environ.get('TWITTER_CLIENT_SECRET')
+  
+    #redirect_uri = 'https://{}/api/logged-in'.format(os.environ.get('PROJECT_DOMAIN') + '.glitch.me')
+    
+    endpoint_url = g.app_url
+    redirect_uri = endpoint_url + og_url_for('.get_loggedin_html')
+    
+    # Set the scopes
+    scopes = TWITTER_SCOPES
+
+    # Create a code verifier
+    code_verifier = base64.urlsafe_b64encode(os.urandom(30)).decode("utf-8")
+    code_verifier = re.sub("[^a-zA-Z0-9]+", "", code_verifier)
+
+    # Create a code challenge
+    code_challenge = hashlib.sha256(code_verifier.encode("utf-8")).digest()
+    code_challenge = base64.urlsafe_b64encode(code_challenge).decode("utf-8")
+    code_challenge = code_challenge.replace("=", "")
+
+    # Start an OAuth 2.0 session
+    oauth = OAuth2Session(client_id, redirect_uri=redirect_uri, scope=scopes)
+
+    # Create an authorize URL
+    auth_url = "https://twitter.com/i/oauth2/authorize"
+    authorization_url, state = oauth.authorization_url(
+        auth_url, code_challenge=code_challenge, code_challenge_method="S256"
+    )
+    
+    session['twitter_code_verifier'] = code_verifier
+    
+    return redirect(authorization_url)
+
+
+
+
+
+
+
+
+
+@twitter_app.route('/refresh-token', methods=['GET'])
+def get_twitter_refresh_token (response_format='json'):
+    
+    client_id = os.environ.get('TWITTER_CLIENT_ID')
+    client_secret = os.environ.get('TWITTER_CLIENT_SECRET')
+    
+    
+    me = request.args.get('me')
+    twitter = session.get(me)
+    
+    if not twitter:
+        return redirect(url_for('.get_login_html'))
+    
+    token = twitter['refresh_token']
+    
+    basic_auth = base64.b64encode('{}:{}'.format(client_id, client_secret).encode('utf-8')).decode('utf-8')
+    
+    headers = {
+        'Authorization': 'Basic ' + basic_auth,
+    }
+    
+    data = {
+        'refresh_token': token,
+        'grant_type': 'refresh_token'
+    }
+    
+    response = requests.post('https://api.twitter.com/2/oauth2/token', data=data, headers=headers)
+    
+    result = json.loads(response.text)
+    
+    if 'access_token' in result:
+         
+        twitter['refresh_token'] = result['refresh_token']
+        twitter['access_token'] = result['access_token']
+        
+        session[ me ] = twitter
+    
+    return response.text
+
+
+
+@twitter_app.route('/app/refresh-token', methods=['GET'])
+def get_twitter_app_refresh_token ():
+    
+    client_id = os.environ.get('TWITTER_CLIENT_ID')
+    client_secret = os.environ.get('TWITTER_CLIENT_SECRET')
+    
+    
+    basic_auth = base64.b64encode('{}:{}'.format(app_consumer_key, app_secret_key).encode('utf-8')).decode('utf-8')
+    
+    headers = {
+        'Authorization': 'Basic ' + basic_auth,
+    }
+    
+    data = {
+        'grant_type': 'client_credentials'
+    }
+    
+    response = requests.post('https://api.twitter.com/oauth2/token', data=data, headers=headers)
+    
+    result = json.loads(response.text)
+    
+    if 'access_token' in result:
+         
+        app_access_token = result['access_token']
+        
+        
+    
+    return response.text
+
+@twitter_app.route('/', methods=['GET'])
+@twitter_app.route('/accounts.html', methods=['GET'])
+def get_loggedin_accounts_html ():
+    twitter_accounts = dict(filter(lambda e: e[0].startswith('twitter:'), session.items()))
+    
+    return Response(json.dumps(twitter_accounts), mimetype='application/json')
+
+
+@twitter_app.route('/tweets/create', methods=['POST'])
+def post_tweets_create ():
+    
+    me = request.args.get('me')
+    twitter = session.get(me)
+    
+    if not twitter:
+        return redirect(url_for('.get_login_html'))
+    
+    user_id = twitter['id']
+    token = twitter['access_token']
+    
+    text = request.form.get('text')
+    reply_to_tweet_id = request.form.get('reply_to_tweet_id')
+    quote_tweet_id = request.form.get('quote_tweet_id')
+    
+    tweet_source = ApiV2TweetSource(token)
+    result = tweet_source.create_tweet(text, reply_to_tweet_id=reply_to_tweet_id, quote_tweet_id=quote_tweet_id)
+    
+    print(result)
+    
+    if 'HX-Request' in request.headers:
+        return render_template('partial/compose-form.html', new_tweet_id=result['data']['id'])
+    else:
+        response_body = json.dumps({
+            'result': result
+        })
+        return Response(response_body, mimetype='application/json')
+
+
+@twitter_app.route('/data/timeline/user/<user_id>/counts')
+def get_data_timeline_user_counts (user_id):
+    query = f'from:{user_id}'
+    
+    # is:reply is:quote is:retweet has:links has:mentions has:media has:images has:videos has:geo
+    
+    if not app_access_token:
+        return 'refresh app token first.', 400
+        
+    
+    tweet_source = ApiV2TweetSource(app_access_token)
+    
+    response_json = tweet_source.count_tweets(query)
+    
+    data = list(filter(lambda d: d.get('tweet_count') > 0, response_json.get('data')))
+    
+    result = {
+        'total_count': response_json.get('meta').get('total_tweet_count'),
+        'data': data
+    }
+    
+    return Response(json.dumps(result), mimetype='application/json')
+
+# ---------------------------------------------------------------------------------------------------------
+# ---------------------------------------------------------------------------------------------------------
+#  HTMx partials
+# ---------------------------------------------------------------------------------------------------------
+# ---------------------------------------------------------------------------------------------------------
+
+
+def tweet_model (includes, tweet_data, me):
+    # retweeted_by, avi_icon_url, display_name, handle, created_at, text
+    
+    
+    user = list(filter(lambda u: u.get('id') == tweet_data['author_id'], includes.get('users')))[0]
+    source_url = 'https://twitter.com/{}/status/{}'.format(user['username'], tweet_data['id'])
+    
+    avi_icon_url = user['profile_image_url']
+    
+    retweet_of = None
+    quoted = None
+    if 'referenced_tweets' in tweet_data:
+        retweet_of = list(filter(lambda r: r['type'] == 'retweeted', tweet_data['referenced_tweets']))
+        quoted = list(filter(lambda r: r['type'] == 'quoted', tweet_data['referenced_tweets']))
+        
+    t = {
+        'id': tweet_data['id'],
+        'text': tweet_data['text'],
+        'created_at': tweet_data['created_at'],
+        'author_is_verified': user['verified'],
+        
+        'conversation_id': tweet_data['conversation_id'],
+        
+        'avi_icon_url': avi_icon_url,
+        
+        'display_name': user['name'],
+        'handle': user['username'],
+        
+        'author_url': url_for('.get_profile_html', user_id=user['id']),
+        'author_id': user['id'],
+        
+        'source_url': source_url,
+        'source_author_url': 'https://twitter.com/{}'.format(user['username']),
+        #'is_edited': len(tweet_data['edit_history_tweet_ids']) > 1
+        
+        
+    }
+    
+    if 'entities' in tweet_data:
+        if 'urls' in tweet_data['entities']:
+            urls = list(filter(lambda u: 'title' in u and 'description' in u, tweet_data['entities']['urls']))
+            
+            if len(urls):
+                url = urls[0]
+                t['card'] = {
+                    'display_url': url['display_url'].split('/')[0],
+                    'source_url': url['unwound_url'],
+                    'content': url['description'],
+                    'title': url['title']
+                }
+    
+    if 'public_metrics' in tweet_data:
+        t['public_metrics'] = tweet_data['public_metrics']
+    
+    if 'non_public_metrics' in tweet_data:
+        t['non_public_metrics'] = tweet_data['non_public_metrics']
+        
+    try:
+        if 'attachments' in tweet_data and 'media_keys' in tweet_data['attachments']:
+
+            
+            media = list(map(lambda mk: list(filter(lambda m: m['media_key'] == mk, includes['media']))[0], tweet_data['attachments']['media_keys']))
+            
+            photos = list(filter(lambda m: m['type'] == 'photo', media))
+            videos = list(filter(lambda m: m['type'] == 'video', media))
+            
+            photos = list(map(lambda p: {**p, 'preview_image_url': p['url'] + '?name=tiny&format=webp'}, photos))
+            videos = list(map(lambda p: {**p, 'image_url': p['preview_image_url'], 'preview_image_url': p['preview_image_url'] + '?name=tiny&format=webp'}, videos))
+            
+            t['photos'] = photos
+            t['videos'] = videos
+    except:
+        print('exception adding attachments to tweet.')
+        
+    if retweet_of and len(retweet_of):
+        retweeted_tweet = list(filter(lambda t: t.get('id') == retweet_of[0]['id'], includes.get('tweets')))[0]
+        
+        t.update({
+            'source_retweeted_by_url': 'https://twitter.com/{}'.format(user['username']),
+            'retweeted_by': user['name'],
+            'retweeted_by_url': '/profile/{}.html'.format(user['id'])
+            })
+        
+        rt = tweet_model(includes, retweeted_tweet, me)
+        t.update(rt)
+    
+    try:
+        if quoted and len(quoted):
+            quoted_tweet = list(filter(lambda t: t.get('id') == quoted[0]['id'], includes.get('tweets')))[0]
+            
+            t['quoted_tweet'] = tweet_model(includes, quoted_tweet, me)
+    except:
+        print('error adding quoted tweet')
+        
+    return t
+
+
+
+def tweet_paginated_timeline ():
+    return
+
+
+
+@twitter_app.route('/data/tweets', methods=['GET'])
+def get_twitter_tweets ():
+
+    me = request.args.get('me')
+    twitter = session.get(me)
+    
+    if not twitter:
+        return redirect(url_for('.get_login_html'))
+    
+    user_id = twitter['id']
+    token = twitter['access_token']
+
+    ids = request.args.get('ids')
+    max_id=''
+    if ids:
+      ids = ids.split(',')
+
+    tweet_source = ApiV2TweetSource(token)
+    
+    response_json = tweet_source.get_tweets(ids)
+    
+    user = {
+            'id': user_id
+        }
+        
+    query = {}
+    
+    if 'HX-Request' in request.headers:
+        
+        includes = response_json.get('includes')
+        tweets = list(map(lambda t: tweet_model(includes, t, me), response_json['data']))
+    
+        return render_template('partial/tweets-timeline.html', user = user, tweets = tweets, query = query)
+    else:
+        return Response(json.dumps(response_json), mimetype="application/json")
+    
+    
+
+@twitter_app.route('/data/timeline/home/<variant>', methods=['GET'])
+def get_data_timeline_home (variant):
+    # retweeted_by, avi_icon_url, display_name, handle, created_at, text
+
+    me = request.args.get('me')
+    twitter = session.get(me)
+    
+    if not twitter:
+        return redirect(url_for('.get_login_html'))
+    
+    user_id = twitter['id']
+    token = twitter['access_token']
+    
+    pagination_token = request.args.get('pagination_token')
+    
+    tweet_source = ApiV2TweetSource(token)
+    response_json = tweet_source.get_home_timeline(user_id,
+                                                    pagination_token = pagination_token)
+    
+    includes = response_json.get('includes')
+    tweets = list(map(lambda t: tweet_model(includes, t, me), response_json['data']))
+    next_token = response_json.get('meta').get('next_token')
+    
+    query = {}
+    
+    if next_token:
+        query = {
+            **query,
+            
+            'next_data_url': url_for('.get_data_timeline_home', variant=variant, pagination_token=next_token)
+        }
+    
+    if 'HX-Request' in request.headers:
+        user = {
+            'id': user_id
+        }
+        
+        return render_template('partial/tweets-timeline.html', user = user, tweets = tweets, query = query)
+    else:
+        response_body = json.dumps({
+            'tweets': tweets,
+            'query': query
+        })
+        return Response(response_body, mimetype='application/json')
+
+
+
+
+@twitter_app.route('/data/mentions/<user_id>', methods=['GET'])
+def get_data_mentions (user_id):
+    
+    me = request.args.get('me')
+    twitter = session.get(me)
+    
+    if not twitter:
+        return redirect(url_for('.get_login_html'))
+    
+    token = twitter['access_token']
+    
+    pagination_token = request.args.get('pagination_token')
+    
+    tweet_source = ApiV2TweetSource(token)
+    response_json = tweet_source.get_mentions_timeline(user_id,
+                                                    pagination_token = pagination_token)
+    
+    # the OG tweet is in the include.tweets collection.
+    # All thread tweets are as well, clearly. Does it cost a fetch?
+    #print(response_json)
+    
+    includes = response_json.get('includes')
+    tweets = list(map(lambda t: tweet_model(includes, t, token), response_json['data']))
+    
+    related_tweets = [] # derived from includes
+    
+    tweets.reverse()
+    
+    next_token = response_json.get('meta').get('next_token')
+    
+    query = {}
+    
+    
+    if next_token:
+        query = {
+            **query,
+            
+            'next_data_url': '/twitter/data/mentions/{}?me={}&pagination_token={}'.format(user_id, me, next_token)
+        }
+    
+    if 'HX-Request' in request.headers:
+        user = {
+            'id': user_id
+        }
+
+        # console.log(res.tweets.map(t => t.text).join("\n\n-\n\n"))
+        return render_template('partial/tweets-timeline.html', user = user, tweets = tweets, query = query, me = me)
+    else:
+        response_body = json.dumps({
+            'tweets': tweets,
+            'pagination_token': pagination_token,
+            'next_token': next_token
+        })
+        return Response(response_body, mimetype='application/json')
+
+
+
+@twitter_app.route('/data/between/<user_id>/<user2_id>', methods=['GET'])
+def get_data_between (user_id, user2_id):
+    
+    me = request.args.get('me')
+    twitter = session.get(me)
+    
+    if not twitter:
+        return redirect(url_for('.get_login_html'))
+
+    token = twitter['access_token']
+    
+    pagination_token = request.args.get('pagination_token')
+    
+    if user_id == 'me':
+        user_id = twitter['id']
+        
+    if user2_id == 'me':
+        user2_id = twitter['id']
+        
+    search_query = "(from:{} to:{}) OR (to:{} from:{})".format(user_id, user2_id, user_id, user2_id)
+    
+    tweet_source = ApiV2TweetSource(token)
+    response_json = tweet_source.search_tweets(search_query,
+                                                    pagination_token = pagination_token)
+    
+    # the OG tweet is in the include.tweets collection.
+    # All thread tweets are as well, clearly. Does it cost a fetch?
+    #print(response_json)
+    
+    # augment with archive if one of the users is me
+    # /twitter-archive/tweets/search?in_reply_to_user_id=__
+    # /twitter-archive/tweets/search?q=@__
+    
+    tweets = []
+    next_token = None
+    if response_json.get('meta').get('result_count'):
+        includes = response_json.get('includes')
+        tweets = list(map(lambda t: tweet_model(includes, t, token), response_json['data']))
+        
+        related_tweets = [] # derived from includes
+        
+        next_token = response_json.get('meta').get('next_token')
+    
+    
+    
+    tweets.reverse()
+    
+    
+    
+    query = {}
+    
+    
+    if next_token:
+        query = {
+            **query,
+            
+            'next_data_url': '/twitter/data/mentions/{}?me={}&pagination_token={}'.format(user_id, me, next_token)
+        }
+    
+    if 'HX-Request' in request.headers:
+        user = {
+            'id': twitter['id']
+        }
+
+        # console.log(res.tweets.map(t => t.text).join("\n\n-\n\n"))
+        return render_template('partial/tweets-timeline.html', user = user, tweets = tweets, query = query, me = me)
+    else:
+        response_body = json.dumps({
+            'tweets': tweets,
+            'pagination_token': pagination_token,
+            'next_token': next_token
+        })
+        return Response(response_body, mimetype='application/json')
+
+
+@twitter_app.route('/data/thread/<tweet_id>', methods=['GET'])
+def get_data_thread (tweet_id):
+    
+    me = request.args.get('me')
+    twitter = session.get(me)
+    
+    if not twitter:
+        return redirect(url_for('.get_login_html'))
+    
+    user_id = twitter['id']
+    token = twitter['access_token']
+    
+    pagination_token = request.args.get('pagination_token')
+    
+    tweet_source = ApiV2TweetSource(token)
+    response_json = tweet_source.get_thread(tweet_id, author_id=user_id,
+                                                    pagination_token = pagination_token)
+    
+    # the OG tweet is in the include.tweets collection.
+    # All thread tweets are as well, clearly. Does it cost a fetch?
+    print(response_json)
+    
+    tweets = []
+    next_token = None
+    if response_json.get('meta').get('result_count'):
+        includes = response_json.get('includes')
+        tweets = list(map(lambda t: tweet_model(includes, t, me), response_json['data']))
+        
+        # FIXME this method is OK except it doesn't work if there are no replies.
+        #tweets.append(tweet_model(includes, list(filter(lambda t: t['id'] == tweet_id, includes.get('tweets')))[0], me))
+        
+        #related_tweets = [] # derived from includes
+        
+        next_token = response_json.get('meta').get('next_token')
+    
+    if not pagination_token:
+        response_json = tweet_source.get_tweet(tweet_id)
+
+        print("parent tweet=")
+        print(response_json)
+        
+        includes = response_json.get('includes')
+        tweet = response_json.get('data')[0]
+        
+        tweets.append(tweet_model(includes, tweet, me))
+    
+    tweets.reverse()
+
+    
+    
+    query = {}
+    
+    if next_token:
+        query = {
+            **query,
+            
+            'next_data_url': '/twitter/data/thread/{}?me={}&pagination_token={}'.format(tweet_id, me, next_token)
+        }
+    
+    if 'HX-Request' in request.headers:
+        user = {
+            'id': user_id
+        }
+
+        # console.log(res.tweets.map(t => t.text).join("\n\n-\n\n"))
+        return render_template('partial/tweets-timeline.html', user = user, tweets = tweets, query = query)
+    else:
+        response_body = json.dumps({
+            'tweets': tweets,
+            'pagination_token': pagination_token,
+            'next_token': next_token
+        })
+        return Response(response_body, mimetype='application/json')
+
+
+
+@twitter_app.route('/data/conversation/<tweet_id>', methods=['GET'])
+def get_data_conversation (tweet_id):
+    
+    me = request.args.get('me')
+    twitter = session.get(me)
+    
+    if not twitter:
+        return redirect(url_for('.get_login_html'))
+    
+    user_id = twitter['id']
+    token = twitter['access_token']
+
+    pagination_token = request.args.get('pagination_token')
+    
+    tweet_source = ApiV2TweetSource(token)
+    
+    # seems to get l
+    response_json = tweet_source.get_thread(tweet_id,
+                                                    pagination_token = pagination_token)
+    
+    # the OG tweet is in the include.tweets collection.
+    # All thread tweets are as well, clearly. Does it cost a fetch?
+    #print(response_json)
+    
+    tweets = []
+    next_token = None
+
+    print("conversation meta:")
+    print(json.dumps(response_json.get('meta'), indent=2))
+
+    if response_json.get('meta').get('result_count'):
+    
+        includes = response_json.get('includes')
+        tweets = list(map(lambda t: tweet_model(includes, t, me), response_json['data']))
+        
+        next_token = response_json.get('meta').get('next_token')
+    
+    # this method is OK except it doesn't work if there are no replies.
+    #tweets.append(tweet_model(includes, list(filter(lambda t: t['id'] == tweet_id, includes.get('tweets')))[0], me))
+    
+    if not pagination_token:
+        response_json = tweet_source.get_tweet(tweet_id)
+        
+        
+        
+        print("parent tweet=")
+        print(response_json)
+        
+        includes = response_json.get('includes')
+        tweet = response_json.get('data')[0]
+        
+        tweets.append(tweet_model(includes, tweet, me))
+    
+    #related_tweets = [] # derived from includes
+    
+    tweets.reverse()
+    
+    
+    
+    query = {}
+    
+    if next_token:
+        query = {
+            **query,
+            'next_data_url': '/twitter/data/conversation/{}?me={}&pagination_token={}'.format(tweet_id, me, next_token)
+        }
+    
+    if 'HX-Request' in request.headers:
+        user = {
+            'id': user_id
+        }
+        
+        # console.log(res.tweets.map(t => t.text).join("\n\n-\n\n"))
+        return render_template('partial/tweets-timeline.html', user = user, tweets = tweets, query = query)
+    else:
+        response_body = json.dumps({
+            'tweets': tweets,
+            'pagination_token': pagination_token,
+            'next_token': next_token
+        })
+        return Response(response_body, mimetype='application/json')
+
+
+
+@twitter_app.route('/data/timeline/user/<user_id>', methods=['GET'])
+def get_data_timeline_user (user_id ):
+    
+    me = request.args.get('me')
+    twitter = session.get(me)
+    
+    if not twitter:
+        return redirect(url_for('.get_login_html'))
+    
+    token = twitter['access_token']
+    
+    pagination_token = request.args.get('pagination_token')
+    exclude_replies = request.args.get('exclude_replies')
+    
+    is_me = user_id == twitter['id']
+    
+    tweet_source = ApiV2TweetSource(token)
+    response_json = tweet_source.get_user_timeline(user_id,
+                                                    pagination_token = pagination_token,
+                                                    non_public_metrics = is_me,
+                                                    exclude_replies = exclude_replies == '1')
+    
+    
+    print(response_json)
+    
+    includes = response_json.get('includes')
+    tweets = list(map(lambda t: tweet_model(includes, t, me), response_json['data']))
+    next_token = response_json.get('meta').get('next_token')
+    
+    query = {}
+    
+    if next_token:
+        query = {
+            **query,
+            
+            'next_data_url': url_for('.get_data_timeline_user', user_id=user_id , pagination_token=next_token)
+        }
+    
+    if 'HX-Request' in request.headers:
+        user = {
+            'id': user_id
+        }
+        
+        return render_template('partial/tweets-timeline.html', user = user, tweets = tweets, query = query)
+    else:
+        response_body = json.dumps({
+            'tweets': tweets,
+            'query': query
+        })
+        return Response(response_body, mimetype='application/json')
+
+
+
+
+
+@twitter_app.route('/data/bookmarks/<user_id>', methods=['GET'])
+def get_data_bookmarks (user_id):
+    # retweeted_by, avi_icon_url, display_name, handle, created_at, text
+    
+    me = request.args.get('me')
+    twitter = session.get(me)
+    
+    if not twitter:
+        return redirect(url_for('.get_login_html'))
+    
+    token = twitter['access_token']
+    
+    pagination_token = request.args.get('pagination_token')
+    
+    tweet_source = ApiV2TweetSource(token)
+    response_json = tweet_source.get_bookmarks(user_id,
+                                                    pagination_token = pagination_token)
+    
+    
+    includes = response_json.get('includes')
+    tweets = list(map(lambda t: tweet_model(includes, t, me), response_json['data']))
+    next_token = response_json.get('meta').get('next_token')
+    
+    query = {}
+    
+    if next_token:
+        query = {
+            **query,
+            
+            'next_data_url': '/twitter/data/bookmarks/{}?me={}&pagination_token={}'.format(user_id, me, next_token),
+            'next_page_url': '?me={}&pagination_token={}'.format(me, next_token)
+        }
+    
+    if 'HX-Request' in request.headers:
+        user = {
+            'id': user_id
+        }
+        
+        return render_template('partial/tweets-timeline.html', user = user, tweets = tweets, query = query)
+    else:
+        response_body = json.dumps({
+            'tweets': tweets,
+            'query': query
+        })
+        return Response(response_body, mimetype='application/json')
+
+
+
+
+
+# ---------------------------------------------------------------------------------------------------------
+# ---------------------------------------------------------------------------------------------------------
+#  HTMx views
+# ---------------------------------------------------------------------------------------------------------
+# ---------------------------------------------------------------------------------------------------------
+
+
+
+
+@twitter_app.route('/latest.html', methods=['GET'])
+def get_timeline_home_html (variant = "reverse_chronological", pagination_token=None):
+    # retweeted_by, avi_icon_url, display_name, handle, created_at, text
+
+    me = request.args.get('me')
+    twitter = session.get(me)
+    
+    if not twitter:
+        return redirect(url_for('.get_login_html'))
+    
+    user_id = twitter['id']
+    token = twitter['access_token']
+    
+    if not pagination_token:
+        pagination_token = request.args.get('pagination_token')
+    
+    tweet_source = ApiV2TweetSource(token)
+    response_json = tweet_source.get_home_timeline(user_id,
+                                                    pagination_token = pagination_token)
+    
+    print(json.dumps(response_json, indent=2))
+    
+    includes = response_json.get('includes')
+    tweets = list(map(lambda t: tweet_model(includes, t, me), response_json['data']))
+    next_token = response_json.get('meta').get('next_token')
+    
+    query = {}
+    
+    if next_token:
+        query = {
+            **query,
+            
+            'next_data_url': url_for('.get_data_timeline_home', variant=variant, pagination_token=next_token),
+            'next_page_url': url_for('.get_timeline_home_html', pagination_token=pagination_token)
+        }
+    
+    user = {
+            'id': user_id
+        }
+    
+    return render_template('tweet-collection.html', user = user, tweets = tweets, query = query)
+
+
+
+
+
+@twitter_app.route('/bookmarks.html', methods=['GET'])
+def get_bookmarks_html (user_id = None, pagination_token=None, token=None):
+    # retweeted_by, avi_icon_url, display_name, handle, created_at, text
+    
+    me = request.args.get('me')
+    twitter = session.get(me)
+    
+    if not twitter:
+        return redirect(url_for('.get_login_html'))
+    
+    if not user_id:
+        user_id = twitter['id']
+    
+    token = twitter['access_token']
+    
+    if not pagination_token:
+        pagination_token = request.args.get('pagination_token')
+    
+    tweet_source = ApiV2TweetSource(token)
+    response_json = tweet_source.get_bookmarks(user_id,
+                                                    pagination_token = pagination_token)
+    
+    print(response_json)
+    
+    includes = response_json.get('includes')
+    tweets = list(map(lambda t: tweet_model(includes, t, me), response_json['data']))
+    next_token = response_json.get('meta').get('next_token')
+    
+    query = {}
+    
+    if next_token:
+        query = {
+            **query,
+            
+            'next_data_url': url_for('.get_data_bookmarks', user_id=user_id, pagination_token=next_token),
+            'next_page_url': url_for('.get_bookmarks_html', user_id=user_id, pagination_token=pagination_token)
+        }
+    
+    user = {
+            'id': user_id
+        }
+    
+    return render_template('tweet-collection.html', user = user, tweets = tweets, query = query)
+
+
+
+
+
+@twitter_app.route('/conversations.html', methods=['GET'])
+def get_conversations_html ():
+    
+    me = request.args.get('me')
+    twitter = session.get(me)
+    
+    if not twitter:
+        return redirect(url_for('.get_login_html'))
+    
+    user_id = twitter['id']
+    token = twitter['access_token']
+    
+    
+    pagination_token = request.args.get('pagination_token')
+    max_results = int(request.args.get('max_results', 10))
+    
+    # https://developer.twitter.com/en/docs/twitter-api/direct-messages/lookup/api-reference/get-dm_events
+    url = "https://api.twitter.com/2/dm_events"
+
+    params = {
+        "dm_event.fields": "id,event_type,text,created_at,dm_conversation_id,sender_id,participant_ids,referenced_tweets,attachments",
+        "expansions": ",".join(["sender_id", "participant_ids"]),
+
+        "max_results": max_results,
+        
+        "user.fields": ",".join(["id", "created_at", "name", "username", "location", "profile_image_url", "url", "verified"])
+    }
+    
+    if pagination_token:
+        params['pagination_token'] = pagination_token
+        
+    headers = {"Authorization": "Bearer {}".format(token)}
+    
+    response = requests.get(url, params=params, headers=headers)
+    response_json = json.loads(response.text)
+    
+    print(response_json)
+    
+    dm_events = response_json.get('data')
+
+    next_token = response_json.get('meta').get('next_token')
+    
+    query = {
+        'pagination_token': pagination_token,
+        'next_token': next_token
+    }
+    
+    user = {
+            'id': user_id
+        }
+    
+    return render_template('conversations.html', user = user, dm_events = dm_events, query = query)
+
+
+
+@twitter_app.route('/profile/<user_id>.html', methods=['GET'])
+def get_profile_html (user_id):
+    
+    me = request.args.get('me')
+    twitter = session.get(me)
+    
+    if not twitter:
+        return redirect(url_for('.get_login_html'))
+    
+    token = twitter['access_token']
+    
+    is_me = user_id == twitter['id']
+    
+    pagination_token = request.args.get('pagination_token')
+    exclude_replies = request.args.get('exclude_replies', '1')
+    
+    tweet_source = ApiV2TweetSource(token)
+    response_json = tweet_source.get_user_timeline(user_id,
+                                                    exclude_replies = exclude_replies == '1',
+                                                    pagination_token = pagination_token,
+                                                    non_public_metrics = is_me)
+    
+    profile_links = []
+    if user_id == "":
+        profile_links += [
+            {
+                'title': 'Mastodon',
+                'type': 'rss',
+                'url': 'https://mastodon.cloud/@ispoogedaily.rss'
+            },
+            {
+                'title': 'YouTube',
+                'type': 'rss',
+                'url': 'https://mastodon.cloud/@ispoogedaily.rss'
+            },
+            {
+                'title': 'Reddit',
+                'type': 'rss',
+                'url': 'https://mastodon.cloud/@ispoogedaily.rss'
+            },
+            {
+                'title': 'BedRSS',
+                'type': 'rss',
+                'url': 'https://mastodon.cloud/@ispoogedaily.rss'
+            },
+            {
+                'title': 'ispoogedaily',
+                'type': 'ig',
+                'url': 'https://mastodon.cloud/@ispoogedaily.rss'
+            },
+            {
+                'title': 'iSpooge Daily',
+                'type': 'yt',
+                'url': 'https://mastodon.cloud/@ispoogedaily.rss'
+            },
+            {
+                'title': 'US 555-555-5555',
+                'type': 'tel',
+                'url': 'tel:+15555555555'
+            },
+            {
+                'title': 'biz@example.com',
+                'type': 'email',
+                'url': 'mailto:biz@example.com?subject=hey'
+            },
+        ]
+    
+    includes = response_json.get('includes')
+    tweets = list(map(lambda t: tweet_model(includes, t, me), response_json['data']))
+    next_token = response_json.get('meta').get('next_token')
+    
+    query = {}
+    
+    if next_token:
+        query = {
+            **query,
+            
+            'next_data_url': url_for('.get_data_timeline_user', user_id=user_id, pagination_token=next_token, exclude_replies=1),
+            'next_page_url': url_for('.get_profile_html', user_id=user_id , pagination_token=next_token)
+        }
+    
+    profile_user = {
+            'id': user_id
+        }
+    
+    return render_template('user-profile.html', user = profile_user, tweets = tweets, query = query)
+
+
+
+
+
+
+@twitter_app.route('/media/upload', methods=['POST'])
+def post_media_upload ():
+    me = request.args.get('me')
+    twitter = session.get(me)
+    
+    if not twitter:
+        return redirect(url_for('.get_login_html'))
+    
+    token = twitter['access_token']
+    form = {
+        'media_category': 'tweet_image'
+    }
+    
+    headers = {
+        'Authorization': 'Bearer {}'.format(token)
+    }
+    
+    url = 'http://localhost:5004/twitter/fake-twitter/media/upload'
+    #url = 'https://upload.twitter.com/1.1/media/upload.json' # .json
+    
+    
+    
+    upload_media = {}
+    for e in request.files.items():
+        media_name = e[0]
+        f = e[1]
+        
+        print('.')
+        
+        files = {'media': [secure_filename(f.filename), BufferedReader(f), f.content_type]}
+
+        response = requests.post(url, files=files, data=form, headers=headers)
+    
+        print(response.status_code)
+        print(response.text)
+    
+        response_json = json.loads(response.text)
+    
+        upload_media[media_name] = response_json
+    
+    return Response(json.dumps({'upload_media': upload_media}), mimetype='application/json')
+
+
+
+@twitter_app.route('/fake-twitter/media/upload', methods=['POST'])
+def post_media_upload2 ():
+    print(request.content_type)
+    
+    f = request.files.get('media')
+    
+    f.seek(0,2)
+    media_size = f.tell()
+
+    media = {
+        #'_auth': request.headers.get('Authorization'),
+        'media_key': '3_{}'.format(secure_filename(f.filename)),
+        'media_id': secure_filename(f.filename),
+        'size': media_size,
+        'expires_after_secs': 86400,
+        'image': {
+            'image_type': f.content_type,
+            'w': 1,
+            'h': 1
+        }
+    }
+
+    return Response(json.dumps(media), mimetype='application/json')
+

Một số tệp đã không được hiển thị bởi vì quá nhiều tập tin thay đổi trong này khác