Record · corpus.blog/posts/01a087af-0ffc-7150-b851-55a6048151c1

ELK-MS - ElasticSearch/LogStash/Kibana - Mesos/Spark : a lightweight and efficient alternative to the Hadoop Stack - part II : assessing behaviour

niceideas.ch · published 23 August 2017

A record is what was published and who pointed at it. The text of the post is not held here: read it at the source, or at its Wayback capture.

Read the post at niceideas.ch Wayback capture

0blogs citing

First cited: —
Most recent: —
Rank this month: not ranked
External links: 20
Words captured: 9,990

Cited by

No blog we hold has cited this post yet.

Links from this post

Link

Host

http://tracker.ceph.com/projects/ceph/wiki/Ceph-mesos

tracker.ceph.com

https://indico.cern.ch/event/524549/contributions/2185930/attachments/1290231/1921189/2016.06.13_-_Spark_on_Ceph.pdf

indico.cern.ch

https://spark.apache.org/docs/latest/running-on-mesos.html

spark.apache.org

https://www.elastic.co/guide/en/elasticsearch/hadoop/current/configuration.html

elastic.co

https://download.elastic.co/demos/kibana/gettingstarted/accounts.zip

download.elastic.co

https://download.elastic.co/demos/kibana/gettingstarted/shakespeare.json

download.elastic.co

https://download.elastic.co/demos/kibana/gettingstarted/logs.jsonl.gz

download.elastic.co

https://issues.apache.org/jira/browse/SPARK-11857

issues.apache.org

https://fr.slideshare.net/SparkSummit/getting-the-best-performance-with-pyspark

fr.slideshare.net

http://aseigneurin.github.io/2016/09/01/spark-calling-scala-code-from-pyspark.html

aseigneurin.github.io

https://www.elastic.co/guide/en/elasticsearch/hadoop/master/spark.html

elastic.co

https://wiki.apache.org/hadoop/MountableHDFS

wiki.apache.org

https://github.com/elastic/elasticsearch-hadoop/issues/276

github.com

https://www.elastic.co/guide/en/elasticsearch/hadoop/current/spark.html

elastic.co

https://spark.apache.org/docs/2.2.0/api/python/pyspark.sql.html

spark.apache.org

https://spark.apache.org/docs/1.6.2/api/python/pyspark.sql.html

spark.apache.org

https://spark.apache.org/docs/latest/configuration.html

spark.apache.org

https://spark.apache.org/docs/latest/monitoring.html

spark.apache.org

https://spark.apache.org/docs/latest/job-scheduling.html

spark.apache.org

https://spark.apache.org/docs/2.2.0/api/python/pyspark.html

spark.apache.org

Elsewhere on niceideas.ch (127)

https://www.niceideas.ch/es_spark/images/application_architecture.png

niceideas.ch

ELK-MS - ElasticSearch/LogStash/Kibana - Mesos/Spark : a lightweight and efficient alternative to the Hadoop Stack - part I : setup the cluster Read at the source ↗

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/niceideas_ELK-MS.tar.gz

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/niceideas_ELK-MS_TEST.tar.gz

niceideas.ch

ELK-MS - ElasticSearch/LogStash/Kibana - Mesos/Spark : a lightweight and efficient alternative to the Hadoop Stack - part III : so why is it cool ? Read at the source ↗

niceideas.ch

https://www.niceideas.ch/es_spark/images/testing.png

niceideas.ch

http://niceideas.ch/mes/swissairbnb/tomslee_airbnb_switzerland_1451_2017-07-11.csv

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/tests/scenarii/1_nominal_1_test_bank_rdd_legacy.sh

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_logs/1_nominal_1_test_bank_rdd_legacy.log

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_cerebro/result_1_1.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_mesos/mesos_fwk_1_1.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_spark/result_1_1.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_spark/result_1_1_job_0_stage_0.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/tests/scenarii/1_nominal_2_test_bank_df_legacy.sh

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_logs/1_nominal_2_test_bank_df_legacy.log

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_cerebro/result_1_2.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_mesos/mesos_fwk_1_2.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_spark/result_1_2.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_spark/result_1_2_job_0_stage_0.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/tests/scenarii/1_nominal_3_test_bank_df.sh

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_logs/1_nominal_3_test_bank_df.log

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_cerebro/result_1_3.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_mesos/mesos_fwk_1_3.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_spark/result_1_3.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_spark/result_1_3_job_0_stage_0.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/tests/scenarii/1_nominal_4_test_apache-logs_df.sh

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_logs/1_nominal_4_test_apache-logs_df.log

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_cerebro/result_1_4.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_mesos/mesos_fwk_1_4.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_spark/result_1_4.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_spark/result_1_4_job_0_stage_0.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/tests/scenarii/1_nominal_5_test_shakespeare.sh

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_logs/1_nominal_5_test_shakespeare.log

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_cerebro/result_1_5.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_mesos/mesos_fwk_1_5.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_spark/result_1_5.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_spark/result_1_5_job_0_stage_0.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/tests/scenarii/2_collocation_1_bank_one_shard.sh

niceideas.ch

https://www.niceideas.ch/es_spark/images/system_archi_data_2_1.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_logs/2_collocation_1_bank_one_shard.log

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_cerebro/result_2_1.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_mesos/mesos_fwk_2_1.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_spark/result_2_1.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_spark/result_2_1_job_0_stage_0.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/tests/scenarii/2_collocation_2_bank_two_shards.sh

niceideas.ch

https://www.niceideas.ch/es_spark/images/system_archi_data_2_2.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_logs/2_collocation_2_bank_two_shards.log

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_cerebro/result_2_2.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_mesos/mesos_fwk_2_2.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_spark/result_2_2.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_spark/result_2_2_job_0_stage_0.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/tests/scenarii/2_collocation_3_bank_three_shards.sh

niceideas.ch

https://www.niceideas.ch/es_spark/images/system_archi_data_2_3.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_logs/2_collocation_3_bank_three_shards.log

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_cerebro/result_2_3.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_mesos/mesos_fwk_2_3.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_spark/result_2_3.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_spark/result_2_3_job_0_stage_0.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/tests/scenarii/2_collocation_4_bank_one_shard_with_replicas.sh

niceideas.ch

https://www.niceideas.ch/es_spark/images/system_archi_data_2_4.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_logs/2_collocation_4_bank_one_shard_with_replicas.log

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_cerebro/result_2_4.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_mesos/mesos_fwk_2_4.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_spark/result_2_4.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_spark/result_2_4_job_0_stage_0.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/tests/scenarii/2_collocation_5_bank_one_shard_repartition_NOT_WORKING.sh

niceideas.ch

https://www.niceideas.ch/es_spark/images/system_archi_data_2_5.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_logs/2_collocation_5_bank_one_shard_repartition_NOT_WORKING.log

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_cerebro/result_2_5.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_mesos/mesos_fwk_2_5.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_spark/result_2_5.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_spark/result_2_5_job_0_stage_0.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/tests/scenarii/3_aggregation_1_es_shakespeare_rdd_legacy_NOT_WORKING.sh

niceideas.ch

https://www.niceideas.ch/es_spark/images/system_archi_data_3_1.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_logs/3_aggregation_1_es_shakespeare_rdd_legacy_NOT_WORKING.log

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_cerebro/result_3_1.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_mesos/mesos_fwk_3_1.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_spark/result_3_1.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_spark/result_3_1_job_0_stage_0.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/tests/scenarii/3_aggregation_2_spark_shakespeare.sh

niceideas.ch

https://www.niceideas.ch/es_spark/images/system_archi_data_3_2.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_logs/3_aggregation_2_spark_shakespeare.log

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_cerebro/result_3_2.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_mesos/mesos_fwk_3_2.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_spark/result_3_2.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_spark/result_3_2_job_0_stage_0.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/tests/scenarii/4_join_1_swissdata_df.sh

niceideas.ch

https://www.niceideas.ch/es_spark/images/system_archi_data_4_1.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_logs/4_join_1_swissdata_df.log

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_cerebro/result_4_1.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_mesos/mesos_fwk_4_1.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_spark/result_4_1.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_spark/result_4_1_job_0_stage_0.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_spark/result_4_1_job_0_stage_1.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/tests/scenarii/5_concurrency_1_swissdata_df.sh

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_logs/5_concurrency_1_swissdata_df.log

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_logs/log_5_concurrency_1_swissdata_1.log

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_logs/log_5_concurrency_1_swissdata_2.log

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_logs/log_5_concurrency_1_swissdata_3.log

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_logs/log_5_concurrency_1_swissdata_4.log

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_mesos/mesos_fwk_5_1_1.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_mesos/mesos_fwk_5_1_2.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_mesos/mesos_fwk_5_1_3.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_mesos/mesos_fwk_5_1_4.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_spark/result_5_1_1.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_spark/result_5_1_2.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_spark/result_5_1_3.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_spark/result_5_1_4.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_spark/result_5_1_1_job_0.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_spark/result_5_1_1_job_0_stage_0.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_spark/result_5_1_1_job_0_stage_1.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_spark/result_5_1_1_job_0_stage_2.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_spark/result_5_1_2_job_0.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_spark/result_5_1_2_job_0_stage_0.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_spark/result_5_1_2_job_0_stage_1.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_spark/result_5_1_2_job_0_stage_2.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_spark/result_5_1_3_job_0.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_spark/result_5_1_3_job_0_stage_0.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_spark/result_5_1_3_job_0_stage_1.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_spark/result_5_1_3_job_0_stage_2.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_spark/result_5_1_4_job_0.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_spark/result_5_1_4_job_0_stage_0.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_spark/result_5_1_4_job_0_stage_1.png

niceideas.ch

https://www.niceideas.ch/es_spark/sandbox/result_spark/result_5_1_4_job_0_stage_2.png

niceideas.ch

https://www.niceideas.ch/es_spark/images/concurrency_performance.png

niceideas.ch

https://www.niceideas.ch/es_spark/images/concurrency_performance_FINE-GRAINED.png

niceideas.ch

http://niceideas.ch/mes/swissairbnb/swisscitiespop.txt

niceideas.ch