英文标题：
《A Short Note on P-Value Hacking》
---
作者：
Nassim Nicholas Taleb
---
最新提交年份：
2018
---
英文摘要：
We present the expected values from p-value hacking as a choice of the minimum p-value among $m$ independents tests, which can be considerably lower than the \"true\" p-value, even with a single trial, owing to the extreme skewness of the meta-distribution. We first present an exact probability distribution (meta-distribution) for p-values across ensembles of statistically identical phenomena. We derive the distribution for small samples $2<n \\leq n^*\\approx 30$ as well as the limiting one as the sample size $n$ becomes large. We also look at the properties of the \"power\" of a test through the distribution of its inverse for a given p-value and parametrization. The formulas allow the investigation of the stability of the reproduction of results and \"p-hacking\" and other aspects of meta-analysis. P-values are shown to be extremely skewed and volatile, regardless of the sample size $n$, and vary greatly across repetitions of exactly same protocols under identical stochastic copies of the phenomenon; such volatility makes the minimum $p$ value diverge significantly from the \"true\" one. Setting the power is shown to offer little remedy unless sample size is increased markedly or the p-value is lowered by at least one order of magnitude.
---
中文摘要：
我们将p值黑客攻击的预期值作为$m$独立测试中最小p值的选择，由于元分布的极端偏斜，该值可能会大大低于“真实”p值，即使是单次试验。我们首先给出了统计上相同现象集合中p值的精确概率分布（元分布）。我们推导了小样本$2<n\\leq n^*\\约30$的分布，以及样本量$n$变大时的极限分布。我们还通过给定p值的逆分布和参数化来研究测试的“幂”性质。这些公式允许调查结果复制的稳定性和“p-hacking”以及元分析的其他方面。结果表明，无论样本大小为$n$，P值都是极为偏斜和不稳定的，并且在相同的随机复制下，完全相同的协议重复之间差异很大；这种波动性使得美元兑便士的最低价值与“真实”价值存在显著差异。结果表明，除非样本量显著增加或p值降低至少一个数量级，否则设置功率几乎不能提供补救措施。
---
分类信息：

一级分类：Statistics 统计学
二级分类：Applications 应用程序
分类描述：Biology, Education, Epidemiology, Engineering, Environmental Sciences, Medical, Physical Sciences, Quality Control, Social Sciences
生物学，教育学，流行病学，工程学，环境科学，医学，物理科学，质量控制，社会科学
--
一级分类：Quantitative Finance 数量金融学
二级分类：Statistical Finance 统计金融
分类描述：Statistical, econometric and econophysics analyses with applications to financial markets and economic data
统计、计量经济学和经济物理学分析及其在金融市场和经济数据中的应用
--

---
PDF下载：
-->