[DeTomaso] Archive searching
Kurt Byrnes
kdb at kbyrnes.com
Thu May 26 14:34:15 EDT 2016
Terry,
Ha ha! We're using pipermail for archive storage. I did a quick look
at Elasticsearch, interesting product, java based. I think it may be
overkill for our needs.
I just installed swish-e and it seems to be working somewhat. I need to
make some changes so modified date is posted date and include the OLD
archives. Also, I have not put a cron job in yet to index periodically
so for the moment, it's searchable through about 2pm EDT. I'll need to
modify the source to clean up the landing page and output search help
stuff but for now, everyone should be able to sort of search the
archives at:
http://server.detomasolist.com/mailman/swish.cgi
Let me know if this is a solution everyone likes.
Kurt
On 5/26/2016 11:36 AM, Himes, Terry (397C) wrote:
> Kurt,
>
> Putting your personal life before others!! How dare you!! ;-))
>
> I know nothing about "mailman" (sorry) but I am working with Docker
> Containers and for Curiosity we
> process (ingest) about 1.2TB of telemetry per week. It takes
> Elasticsearch less than a second to query
> the entire data-store. Both Elasticsearch and Dockers are free (I
> think. I'll check on it) so maybe we can
> run a local copy. AWS and the cloud are not cheap.
>
> Oh! I just googled mailman. Interesting. Mostly written in Python.
> Awesome. Do you use Pipermail or
> HyperKitty for archival storage??
>
>
> Terry
>
>
> /
> /
>
> /
> /
>
> /"//A Purple Heart proves you were smart enough to hatch a// plan,/
>
> / stupid enough to try it //and lucky enough to survive!//"/
>
>
> *Terry W. Himes *
>
> *JPL** Jet Propulsion Laboratory*
>
> *Dawn Spacecraft Team*
>
> *Rosetta Sequence Team Lead*
>
> Phone: (818) 393-6261
>
> Cell: (818) 653-8213
>
> Fax: (818) 393-3147
>
> *thimes at jpl.nasa.gov <mailto:thimes at jpl.nasa.gov>*
>
> 🇺🇸
>
>
> From: DeTomaso <detomaso-bounces at server.detomasolist.com
> <mailto:detomaso-bounces at server.detomasolist.com>> on behalf of Kurt
> Byrnes <kdb at kbyrnes.com <mailto:kdb at kbyrnes.com>>
> Date: Thursday, May 26, 2016 6:43 AM
> To: "DeTomaso at server.detomasolist.com
> <mailto:DeTomaso at server.detomasolist.com>"
> <DeTomaso at server.detomasolist.com
> <mailto:DeTomaso at server.detomasolist.com>>
> Subject: [DeTomaso] Archive searching
>
> I've started looking into providing search capabilities to the
> archives. There are several utilities that have been tied into mailman
> that I'll look into.
>
>
> I'll get things setup so google can crawl the archives however I think
> they only crawl once a week so recent data wouldn't be available.
>
>
> I'm a bit swamped with real work so please be patient.
>
> _______________________________________________
>
>
> Detomaso Forum NO LONGER Managed by POCA
> Posted emails must not exceed 1.5 Megabytes
> DeTomaso mailing list
> DeTomaso at server.detomasolist.com <mailto:DeTomaso at server.detomasolist.com>
> http://server.detomasolist.com/mailman/listinfo/detomaso
>
> To manage your subscription (change email address, unsubscribe, etc.)
> use the links above.
>
> Members who post to this list grant license to the list to forward any
> message posted here to all past, current, or future members of the
> list. They also grant the list owner permission to maintain an archive
> or approve the archiving of list messages.
>
-------------- next part --------------
Terry,
Ha ha! We're using pipermail for archive storage. I did a quick look
at Elasticsearch, interesting product, java based. I think it may be
overkill for our needs.
I just installed swish-e and it seems to be working somewhat. I need
to make some changes so modified date is posted date and include the
OLD archives. Also, I have not put a cron job in yet to index
periodically so for the moment, it's searchable through about 2pm EDT.
I'll need to modify the source to clean up the landing page and output
search help stuff but for now, everyone should be able to sort of
search the archives at:
[1]http://server.detomasolist.com/mailman/swish.cgi
Let me know if this is a solution everyone likes.
Kurt
On 5/26/2016 11:36 AM, Himes, Terry (397C) wrote:
Kurt,
Putting your personal life before others!! How dare you!! ;-))
I know nothing about "mailman" (sorry) but I am working with Docker
Containers and for Curiosity we
process (ingest) about 1.2TB of telemetry per week. It takes
Elasticsearch less than a second to query
the entire data-store. Both Elasticsearch and Dockers are free (I
think. I'll check on it) so maybe we can
run a local copy. AWS and the cloud are not cheap.
Oh! I just googled mailman. Interesting. Mostly written in Python.
Awesome. Do you use Pipermail or
HyperKitty for archival storage??
Terry
"A Purple Heart proves you were smart enough to hatch a plan,
stupid enough to try it and lucky enough to survive!"
Terry W. Himes
JPL Jet Propulsion Laboratory
Dawn Spacecraft Team
Rosetta Sequence Team Lead
Phone: (818) 393-6261
Cell: (818) 653-8213
Fax: (818) 393-3147
[2]thimes at jpl.nasa.gov
From: DeTomaso <[3]detomaso-bounces at server.detomasolist.com> on behalf
of Kurt Byrnes <[4]kdb at kbyrnes.com>
Date: Thursday, May 26, 2016 6:43 AM
To: "[5]DeTomaso at server.detomasolist.com"
<[6]DeTomaso at server.detomasolist.com>
Subject: [DeTomaso] Archive searching
I've started looking into providing search capabilities to the
archives. There are several utilities that have been tied into mailman
that I'll look into.
I'll get things setup so google can crawl the archives however I think
they only crawl once a week so recent data wouldn't be available.
I'm a bit swamped with real work so please be patient.
_______________________________________________
Detomaso Forum NO LONGER Managed by POCA
Posted emails must not exceed 1.5 Megabytes
DeTomaso mailing list
[7]DeTomaso at server.detomasolist.com
[8]http://server.detomasolist.com/mailman/listinfo/detomaso
To manage your subscription (change email address, unsubscribe, etc.)
use the links above.
Members who post to this list grant license to the list to forward any
message posted here to all past, current, or future members of the
list. They also grant the list owner permission to maintain an archive
or approve the archiving of list messages.
References
Visible links
1. http://server.detomasolist.com/mailman/swish.cgi
2. mailto:thimes at jpl.nasa.gov
3. mailto:detomaso-bounces at server.detomasolist.com
4. mailto:kdb at kbyrnes.com
5. mailto:DeTomaso at server.detomasolist.com
6. mailto:DeTomaso at server.detomasolist.com
7. mailto:DeTomaso at server.detomasolist.com
8. http://server.detomasolist.com/mailman/listinfo/detomaso
Hidden links:
10. mailto:thimes at jpl.nasa.gov
11. mailto:detomaso-bounces at server.detomasolist.com
12. mailto:DeTomaso at server.detomasolist.com
More information about the DeTomaso
mailing list