In the history of mathematics, there is no lack of debate over the credibility of statistical justifications. Berkeley’s prolonged review of the techniques of the calculus in The Analyst (1734) is one example. Another is the “vibrating string controversy” among Leonhard Euler, Jean d’Alembert, and Daniel Bernoulli, hinging on whether an “arbitrary” continuous function on a real interval could be represented by a trigonometric sequence. Carl Friedrich Gauss is usually acknowledged with offering the first appropriate evidence of the essential theorem of geometry, saying that every non-constant polynomial over the complicated figures has a root, in his doctorate thesis of 1799; but the history of that theorem is especially knotty, since it was not originally obvious what techniques could properly be used to set up the existence of the roots in question. In the same way, when Gauss provided his evidence of the law of quadratic reciprocity in his Disquitiones Arithmeticae (1801), he started with the statement that Legendre’s claimed evidence a few years before contained a serious gap.
Mathematicians have always been reflectively aware of their techniques, and, as evidence increased more complicated in the Nineteenth century, specialized mathematicians became more precise in focusing the part of rigor. This is obvious in, for example, Carl Jacobi’s compliment of Johann Chris Gustav Lejune Dirichlet: “Dirichlet alone, not I, nor Cauchy, nor Gauss knows what a completely extensive statistical evidence is. Rather we understand it first from him. When Gauss says that he has shown something, it is very clear; when Cauchy says it, one can bet as much pro as con; when Dirichlet says it, it is certain…” (quoted by Schubring21).
Mathematics has, at crucial junctures, designed in more speculative methods. But these times are usually followed by corresponding times of retrenchment, examining fundamentals and progressively implementing a tight deductive design, either to take care of obvious issues or just to make the content simpler to educate convincingly.