[DeTomaso] Archive searching

Kurt Byrnes kdb at kbyrnes.com
Thu May 26 14:34:15 EDT 2016


Terry,

Ha ha!  We're using pipermail for archive storage.  I did a quick look 
at Elasticsearch, interesting product, java based.  I think it may be 
overkill for our needs.

I just installed swish-e and it seems to be working somewhat.  I need to 
make some changes so modified date is posted date and include the OLD 
archives.  Also, I have not put a cron job in yet to index periodically 
so for the moment, it's searchable through about 2pm EDT.  I'll need to 
modify the source to clean up the landing page and output search help 
stuff but for now, everyone should be able to sort of search the 
archives at:

http://server.detomasolist.com/mailman/swish.cgi

Let me know if this is a solution everyone likes.

Kurt

On 5/26/2016 11:36 AM, Himes, Terry (397C) wrote:
> Kurt,
>
> Putting your personal life before others!!  How dare you!!   ;-))
>
> I know nothing about "mailman" (sorry) but I am working with Docker 
> Containers and for Curiosity we
> process (ingest) about 1.2TB of telemetry per week.  It takes 
> Elasticsearch less than a second to query
> the entire data-store.  Both Elasticsearch and Dockers are free (I 
> think. I'll check on it) so maybe we can
> run a local copy.  AWS and the cloud are not cheap.
>
> Oh!  I just googled mailman.  Interesting.  Mostly written in Python. 
>  Awesome.  Do you use Pipermail or
> HyperKitty for archival storage??
>
>
> Terry
>
>
> /
> /
>
> /
> /
>
> /"//A Purple Heart proves you were smart enough to hatch a// plan,/
>
> / stupid enough to try it //and lucky enough to survive!//"/
>
>
> *Terry W. Himes *
>
> *JPL** Jet Propulsion Laboratory*
>
> *Dawn Spacecraft Team*
>
> *Rosetta Sequence Team Lead*
>
> Phone: (818) 393-6261
>
> Cell:     (818) 653-8213
>
> Fax:     (818) 393-3147
>
> *thimes at jpl.nasa.gov <mailto:thimes at jpl.nasa.gov>*
>
> 🇺🇸
>
>
> From: DeTomaso <detomaso-bounces at server.detomasolist.com 
> <mailto:detomaso-bounces at server.detomasolist.com>> on behalf of Kurt 
> Byrnes <kdb at kbyrnes.com <mailto:kdb at kbyrnes.com>>
> Date: Thursday, May 26, 2016 6:43 AM
> To: "DeTomaso at server.detomasolist.com 
> <mailto:DeTomaso at server.detomasolist.com>" 
> <DeTomaso at server.detomasolist.com 
> <mailto:DeTomaso at server.detomasolist.com>>
> Subject: [DeTomaso] Archive searching
>
> I've started looking into providing search capabilities to the
> archives.  There are several utilities that have been tied into mailman
> that I'll look into.
>
>
> I'll get things setup so google can crawl the archives however I think
> they only crawl once a week so recent data wouldn't be available.
>
>
> I'm a bit swamped with real work so please be patient.
>
> _______________________________________________
>
>
> Detomaso Forum NO LONGER Managed by POCA
> Posted emails must not exceed 1.5 Megabytes
> DeTomaso mailing list
> DeTomaso at server.detomasolist.com <mailto:DeTomaso at server.detomasolist.com>
> http://server.detomasolist.com/mailman/listinfo/detomaso
>
> To manage your subscription (change email address, unsubscribe, etc.) 
> use the links above.
>
> Members who post to this list grant license to the list to forward any 
> message posted here to all past, current, or future members of the 
> list. They also grant the list owner permission to maintain an archive 
> or approve the archiving of list messages.
>

-------------- next part --------------
   Terry,
   Ha ha!  We're using pipermail for archive storage.  I did a quick look
   at Elasticsearch, interesting product, java based.  I think it may be
   overkill for our needs.
   I just installed swish-e and it seems to be working somewhat.  I need
   to make some changes so modified date is posted date and include the
   OLD archives.  Also, I have not put a cron job in yet to index
   periodically so for the moment, it's searchable through about 2pm EDT.
   I'll need to modify the source to clean up the landing page and output
   search help stuff but for now, everyone should be able to sort of
   search the archives at:
   [1]http://server.detomasolist.com/mailman/swish.cgi
   Let me know if this is a solution everyone likes.
   Kurt
   On 5/26/2016 11:36 AM, Himes, Terry (397C) wrote:

   Kurt,
   Putting your personal life before others!!  How dare you!!   ;-))
   I know nothing about "mailman" (sorry) but I am working with Docker
   Containers and for Curiosity we
   process (ingest) about 1.2TB of telemetry per week.  It takes
   Elasticsearch less than a second to query
   the entire data-store.  Both Elasticsearch and Dockers are free (I
   think. I'll check on it) so maybe we can
   run a local copy.  AWS and the cloud are not cheap.
   Oh!  I just googled mailman.  Interesting.  Mostly written in Python.
   Awesome.  Do you use Pipermail or
   HyperKitty for archival storage??
   Terry

   "A Purple Heart proves you were smart enough to hatch a plan,

    stupid enough to try it and lucky enough to survive!"

   Terry W. Himes

   JPL Jet Propulsion Laboratory

   Dawn Spacecraft Team

   Rosetta Sequence Team Lead

   Phone: (818) 393-6261

   Cell:     (818) 653-8213

   Fax:     (818) 393-3147

   [2]thimes at jpl.nasa.gov

   From: DeTomaso <[3]detomaso-bounces at server.detomasolist.com> on behalf
   of Kurt Byrnes <[4]kdb at kbyrnes.com>
   Date: Thursday, May 26, 2016 6:43 AM
   To: "[5]DeTomaso at server.detomasolist.com"
   <[6]DeTomaso at server.detomasolist.com>
   Subject: [DeTomaso] Archive searching

   I've started looking into providing search capabilities to the
   archives.  There are several utilities that have been tied into mailman
   that I'll look into.
   I'll get things setup so google can crawl the archives however I think
   they only crawl once a week so recent data wouldn't be available.
   I'm a bit swamped with real work so please be patient.
   _______________________________________________
   Detomaso Forum NO LONGER Managed by POCA
   Posted emails must not exceed 1.5 Megabytes
   DeTomaso mailing list
   [7]DeTomaso at server.detomasolist.com
   [8]http://server.detomasolist.com/mailman/listinfo/detomaso
   To manage your subscription (change email address, unsubscribe, etc.)
   use the links above.
   Members who post to this list grant license to the list to forward any
   message posted here to all past, current, or future members of the
   list. They also grant the list owner permission to maintain an archive
   or approve the archiving of list messages.

References

   Visible links
   1. http://server.detomasolist.com/mailman/swish.cgi
   2. mailto:thimes at jpl.nasa.gov
   3. mailto:detomaso-bounces at server.detomasolist.com
   4. mailto:kdb at kbyrnes.com
   5. mailto:DeTomaso at server.detomasolist.com
   6. mailto:DeTomaso at server.detomasolist.com
   7. mailto:DeTomaso at server.detomasolist.com
   8. http://server.detomasolist.com/mailman/listinfo/detomaso

   Hidden links:
  10. mailto:thimes at jpl.nasa.gov
  11. mailto:detomaso-bounces at server.detomasolist.com
  12. mailto:DeTomaso at server.detomasolist.com


More information about the DeTomaso mailing list