注册
登录
新闻动态
其他科技
返回
使用机器学习发现ElasticSearch性能峰值
作者:
糖果
发布时间:
2024-03-13 03:05:30 (16天前)
来源:
stormforge.io
Using Machine Learning to Find the ElasticSearch Performance Peak October 12, 2020By Steph Rifai mountain scape How do you know where the highest peak of a mountain range is? Well, probably through a quick google search. But in the absence of the internet, you pick a random place to start, climb to the top and look around (Okay, yes you can use a helicopter, plane or maybe even a drone, but stay with me a second). Now that you are at the top, you see some of the mountains are still taller and make an informed decision on where to go next. Climb down, go to the next mountain and climb up again. Look around, see there are still taller peaks and try again. Tired yet? 您怎么知道山脉的最高峰在哪里? 好吧,可能是通过快速的Google搜索。 但是在没有互联网的情况下,您会选择一个随机的地方开始,爬到顶部并环顾四周(好吧,是的,您可以使用直升机,飞机甚至是无人机,但请和我呆一秒钟)。 现在您已经到达山顶,您会看到一些山还更高,并就下一步的去向做出明智的决定。 爬下来,到下一座山再爬上去。 环顾四周,看看仍然有更高的山峰,然后再试一次。 累了吗? The truth is, no one has time to climb all of the mountains to find out which one is the tallest, and certainly no one has time to try different parameter combinations to reach the highest performance of their application (…you knew we’d get here). So how do you know when you’ve got the best parameters for your application? If you’re anything like us, most of the time you try a lot of different options, and once you find one that performs pretty well, you keep it. Behavioral economists call this satisficing. When the trade-off is more time spent searching for a better configuration, it’s better to stop when you’ve crossed a reasonable threshold. That’s why we built the StormForge Platform, to help you find the tallest mountain peak without ever climbing a mountain yourself. 事实是,没有人有时间爬所有的山峰来找出哪一个是最高的,当然没有人有时间尝试不同的参数组合以达到其应用程序的最高性能(…您知道我们会得到这里)。那么,如何知道何时拥有适合您的应用程序的最佳参数呢?如果您像我们一样,大多数时候,您会尝试许多不同的选择,一旦找到一个效果不错的选择,就保留它。行为经济学家称之为 令人满意。如果要权衡更多的时间来寻找更好的配置,则最好在达到合理阈值后停止。因此,我们构建了StormForge平台,以帮助您找到最高的山峰而无需亲自爬山。 When the trade-off is more time spent searching for a better configuration, it’s better to stop when you’ve crossed a reasonable threshold. 如果要权衡更多的时间来寻找更好的配置,则最好在达到合理阈值时停止。 We put our machine learning (ML) to the test against the preset values in the ElasticSearch Helm chart and compared the performance and cost to the configurations our ML found (view our Elasticsearch example “recipe” here). Performance is measured as the duration it takes to process a dataset configured with the rally benchmarking tool (in seconds). For our example, cost is specific to Google Kubernetes Engine, measured as the total cost of CPU and memory per month. Here’s what we got: 我们根据ElasticSearch Helm图表中的预设值对机器学习(ML)进行了测试 ,并将性能和成本与ML找到的配置进行了比较(请在此处查看我们的Elasticsearch示例“食谱” )。绩效的衡量标准是处理使用反弹基准测试工具配置的数据集所花费的持续时间 (以秒为单位)。在我们的示例中,费用特定于Google Kubernetes Engine,以每月CPU和内存的总费用来衡量。这是我们得到的: The above graphic shows the cost and duration measurements for each configuration the ML tried. The ML searched the parameter space, trying 80 different configurations of four parameters (memory, CPU, heap size, and replicas). The triangle represents the helm default, while the pink dots represent the best options of all the configurations the ML tried. Unstable configurations are parameter combinations that prevented the application from deploying, in this case it was 28% of trials. Each trial helped inform where to search next, ultimately giving us 16 best options to choose from. Since we are optimizing for more than one metric, there is not a single “peak”, but at a given cost, there is a single best duration. 上图显示了ML尝试的每种配置的成本和工期测量。ML搜索参数空间,尝试对四个参数(内存,CPU,堆大小和副本)进行80种不同的配置。三角形代表头盔默认值,粉红色点代表ML尝试的所有配置的最佳选择。不稳定的配置是参数组合,阻止了应用程序的部署,在这种情况下,占28%的试用期。每个试验都有助于告知下一步搜索的方向,最终为我们提供16种最佳选择。由于我们针对多个指标进行了优化,因此没有一个“峰值”,但是在给定的成本下,只有一个最佳持续时间。 The helm configuration had a duration of 864 seconds and a cost of $85 per month. Point A has a duration of 628 seconds and cost of $76 per month. So that’s a 10% reduction in cost with a 8% boost in performance. Point B has a duration of 720 seconds and a cost of $81 per month, meaning for almost no change in cost you can get a 17% improvement in performance. If you are willing to increase your budget a bit, Point C costs $92/month and gives you 32% improvement of performance (589ms). Not bad options to have without ever changing a configuration yourself. 头盔配置的持续时间为864秒,每月费用为85美元。点A的持续时间为628秒,每月费用为$ 76。因此,成本降低了10%,性能提高了8%。B点的持续时间为720秒,每月费用为$ 81,这意味着成本几乎没有变化,您可以获得17%的性能提升。如果您愿意稍微增加预算,则Point C的费用为每月$ 92,可让您提高32%的性能(589毫秒)。无需更改配置即可拥有的不错选择。
收藏
举报
1 条回复
动动手指,沙发就是你的了!
登录
后才能参与评论