Terry, Ha ha! We're using pipermail for archive storage. I did a quick look at Elasticsearch, interesting product, java based. I think it may be overkill for our needs. I just installed swish-e and it seems to be working somewhat. I need to make some changes so modified date is posted date and include the OLD archives. Also, I have not put a cron job in yet to index periodically so for the moment, it's searchable through about 2pm EDT. I'll need to modify the source to clean up the landing page and output search help stuff but for now, everyone should be able to sort of search the archives at: http://server.detomasolist.com/mailman/swish.cgi Let me know if this is a solution everyone likes. Kurt On 5/26/2016 11:36 AM, Himes, Terry (397C) wrote:
Kurt,
Putting your personal life before others!! How dare you!! ;-))
I know nothing about "mailman" (sorry) but I am working with Docker Containers and for Curiosity we process (ingest) about 1.2TB of telemetry per week. It takes Elasticsearch less than a second to query the entire data-store. Both Elasticsearch and Dockers are free (I think. I'll check on it) so maybe we can run a local copy. AWS and the cloud are not cheap.
Oh! I just googled mailman. Interesting. Mostly written in Python. Awesome. Do you use Pipermail or HyperKitty for archival storage??
Terry
/ /
/ /
/"//A Purple Heart proves you were smart enough to hatch a// plan,/
/ stupid enough to try it //and lucky enough to survive!//"/
*Terry W. Himes *
*JPL** Jet Propulsion Laboratory*
*Dawn Spacecraft Team*
*Rosetta Sequence Team Lead*
Phone: (818) 393-6261
Cell: (818) 653-8213
Fax: (818) 393-3147
*thimes@jpl.nasa.gov <mailto:thimes@jpl.nasa.gov>*
🇺🇸
From: DeTomaso <detomaso-bounces@server.detomasolist.com <mailto:detomaso-bounces@server.detomasolist.com>> on behalf of Kurt Byrnes <kdb@kbyrnes.com <mailto:kdb@kbyrnes.com>> Date: Thursday, May 26, 2016 6:43 AM To: "DeTomaso@server.detomasolist.com <mailto:DeTomaso@server.detomasolist.com>" <DeTomaso@server.detomasolist.com <mailto:DeTomaso@server.detomasolist.com>> Subject: [DeTomaso] Archive searching
I've started looking into providing search capabilities to the archives. There are several utilities that have been tied into mailman that I'll look into.
I'll get things setup so google can crawl the archives however I think they only crawl once a week so recent data wouldn't be available.
I'm a bit swamped with real work so please be patient.
_______________________________________________
Detomaso Forum NO LONGER Managed by POCA Posted emails must not exceed 1.5 Megabytes DeTomaso mailing list DeTomaso@server.detomasolist.com <mailto:DeTomaso@server.detomasolist.com> http://server.detomasolist.com/mailman/listinfo/detomaso
To manage your subscription (change email address, unsubscribe, etc.) use the links above.
Members who post to this list grant license to the list to forward any message posted here to all past, current, or future members of the list. They also grant the list owner permission to maintain an archive or approve the archiving of list messages.
Terry, Ha ha! We're using pipermail for archive storage. I did a quick look at Elasticsearch, interesting product, java based. I think it may be overkill for our needs. I just installed swish-e and it seems to be working somewhat. I need to make some changes so modified date is posted date and include the OLD archives. Also, I have not put a cron job in yet to index periodically so for the moment, it's searchable through about 2pm EDT. I'll need to modify the source to clean up the landing page and output search help stuff but for now, everyone should be able to sort of search the archives at: [1]http://server.detomasolist.com/mailman/swish.cgi Let me know if this is a solution everyone likes. Kurt On 5/26/2016 11:36 AM, Himes, Terry (397C) wrote: Kurt, Putting your personal life before others!! How dare you!! ;-)) I know nothing about "mailman" (sorry) but I am working with Docker Containers and for Curiosity we process (ingest) about 1.2TB of telemetry per week. It takes Elasticsearch less than a second to query the entire data-store. Both Elasticsearch and Dockers are free (I think. I'll check on it) so maybe we can run a local copy. AWS and the cloud are not cheap. Oh! I just googled mailman. Interesting. Mostly written in Python. Awesome. Do you use Pipermail or HyperKitty for archival storage?? Terry "A Purple Heart proves you were smart enough to hatch a plan, stupid enough to try it and lucky enough to survive!" Terry W. Himes JPL Jet Propulsion Laboratory Dawn Spacecraft Team Rosetta Sequence Team Lead Phone: (818) 393-6261 Cell: (818) 653-8213 Fax: (818) 393-3147 [2]thimes@jpl.nasa.gov From: DeTomaso <[3]detomaso-bounces@server.detomasolist.com> on behalf of Kurt Byrnes <[4]kdb@kbyrnes.com> Date: Thursday, May 26, 2016 6:43 AM To: "[5]DeTomaso@server.detomasolist.com" <[6]DeTomaso@server.detomasolist.com> Subject: [DeTomaso] Archive searching I've started looking into providing search capabilities to the archives. There are several utilities that have been tied into mailman that I'll look into. I'll get things setup so google can crawl the archives however I think they only crawl once a week so recent data wouldn't be available. I'm a bit swamped with real work so please be patient. _______________________________________________ Detomaso Forum NO LONGER Managed by POCA Posted emails must not exceed 1.5 Megabytes DeTomaso mailing list [7]DeTomaso@server.detomasolist.com [8]http://server.detomasolist.com/mailman/listinfo/detomaso To manage your subscription (change email address, unsubscribe, etc.) use the links above. Members who post to this list grant license to the list to forward any message posted here to all past, current, or future members of the list. They also grant the list owner permission to maintain an archive or approve the archiving of list messages. References Visible links 1. http://server.detomasolist.com/mailman/swish.cgi 2. mailto:thimes@jpl.nasa.gov 3. mailto:detomaso-bounces@server.detomasolist.com 4. mailto:kdb@kbyrnes.com 5. mailto:DeTomaso@server.detomasolist.com 6. mailto:DeTomaso@server.detomasolist.com 7. mailto:DeTomaso@server.detomasolist.com 8. http://server.detomasolist.com/mailman/listinfo/detomaso Hidden links: 10. mailto:thimes@jpl.nasa.gov 11. mailto:detomaso-bounces@server.detomasolist.com 12. mailto:DeTomaso@server.detomasolist.com