Jtdylktuy

Currently I am doing an introduction to parameter estimation and fitting probability distributions to sets of data. So in a small synopsis what I understand the whole process to be like is the following:

1) We collect a large amount of raw data, which comes from an underlying probability distribution. We then "graph" the data (perhaps in the form of a bar chart or something similar at least in the 2D and 3D cases).

2) Observing this visual presentation we go through our list of existing probability distributions and form an opinion on which distribution appears to fit the data most precisely.

3) We then take a large sample from this data and attempt to estimate the parameters of our chosen probability distribution by using the array of techinques available at our disposal.

I have a few questions:

i) Is the outline above the procedure used to get parameter estimates?

ii) (more important) What stops us from making any function a probability distribution? What I mean is we have this visual representation of the data, perhaps none of the known probability distributions that we have presently align with the data. What stops us from just saying "this continuous function will now be a distribution as long as it satisfies the necessary axioms." Is there something more rigourous to this? (perhpas I just haven't arrived there yet in my studies).

1) We collect a large amount of raw data, which comes from an
underlying probability distribution. We then "graph" the data (perhaps
in the form of a bar chart or something similar at least in the 2D and
3D cases).

If the data has more then two dimensions (usually the case), then you cannot graph it. You can graph only the marginal distribution, but not the joint distribution.

2) Observing this visual presentation we go through our list of
existing probability distributions and form an opinion on which
distribution appears to fit the data most precisely.

No. First of all, as stated above, graphs don't tell you the whole story. Second, many distributions can look very similar. Third, there is no such a thing as "list of existing distributions". You can go through the list of popular distributions, but the list of all possible distributions is infinite (you can come up with your own distribution, you can define mixtures of any number of any distributions -- this alone makes the list infinite).

Usually based on what you know about the data (given plots, summary statistics, knowledge on what the data represents, how it was collected) you choose some distribution or few distributions that make sense for this data. For example, if it is a count of independent binary things in fixed number of trials, then most likely you will be using binomial distribution. To understand better when what distributions make sense, you can check the Statistics 110 lectures by Joe Blitzstein.

Moreover, even if you would try several different distributions, then you wouldn't do it based on how the data looks, but rather based on model fit statistics (see questions tagged as model-selection).

3) We then take a large sample from this data and attempt to estimate
the parameters of our chosen probability distribution by using the
array of techinques available at our disposal.

Generally yes, if possible.

ii) (more important) What stops us from making any function a
probability distribution? What I mean is we have this visual
representation of the data, perhaps none of the known probability
distributions that we have presently align with the data. What stops
us from just saying "this continuous function will now be a
distribution as long as it satisfies the necessary axioms." Is there
something more rigourous to this? (perhpas I just haven't arrived
there yet in my studies).

If the function follows the mathematical definition of probability density function, or probability mass function, then it is the function. Usually it is not about finding the distribution that looks exactly like your data. If you wanted it to look like your data, then you would use empirical distribution function or things like kernel density, that would look exactly like your data. In most cases we choose simpler distributions, that look approximately like the empirical distribution. We use the distributions to build simplified models of reality, that can be extended beyond the data you collected. Saying it differently, you don't want the distribution to overfitt to your data.

Here you can find example: What is meant by using a probability distribution to model the output data for a regression problem?