Tech talk: A practical introduction to Bayesian hierarchical modelling

  Переглядів 20,241

Faculty

Faculty

День тому

When the data that you’re modelling naturally splits into sectors - like countries, branches of a store, or different hospitals within a region - it’s difficult to decide whether you should model jointly or separately. Modelling jointly takes advantage of all the data available, but ignores the subtleties that distinguish each individual sector. On the other hand, fitting a separate model to each sector is prone to overfitting, especially where there’s limited data.
However, an approach called hierarchical modelling can combine the best of both approaches, allowing us to take advantage of the overarching information across sectors without giving up on their distinctive features.
In this talk, Faculty Data Scientist Omar Sosa will provide an introduction to the approach, focusing heavily on its practical side. He’ll cover:
- When hierarchical modelling can be used.
- How to implement hierarchical modelling.
- The limitations of using this approach.
- What can we learn by implementing hierarchical modelling.
You’ll need some familiarity with Bayesian inference to get the most value from this talk.

КОМЕНТАРІ: 18
@mohammedalmarakby1221
@mohammedalmarakby1221 5 місяців тому
hands down the best explanation for hierarchical modelling on youtube, especially with the graph visualising the effect of pooling strength on the parameter values
@alejdiazdelao8153
@alejdiazdelao8153 Рік тому
Probably the clearest explanation of hierarchical models I’ve ever seen. Great video!
@Olimorveu
@Olimorveu 7 місяців тому
One of the few well explained examples
@goshirago
@goshirago 2 роки тому
Thank you for this well-paced video with its explanations. I feel much more confident in my understanding of Bayesian hierarchical modelling.
@hemman7931
@hemman7931 26 днів тому
50:20 Make sure you know when to use variational inference instead of MCMC. (usually when working with large datasets)
@logitfau252
@logitfau252 7 днів тому
great one, currently looking for overlap between same treatment used for different diseases, this one looks very helpful to approach for synthesize
@gavinaustin4474
@gavinaustin4474 2 роки тому
Thanks Omar. Clear and helpful.
@juluribk
@juluribk 2 роки тому
Thanks for clear explanation. Very helpful.
@kaushikgupta1410
@kaushikgupta1410 Рік тому
Perhaps the best explanation. Thanks a hell lot
@piotrlukasinski4063
@piotrlukasinski4063 Рік тому
Amazing lecture!
@user-wr4yl7tx3w
@user-wr4yl7tx3w 2 роки тому
this is really well explained.
@maraffio72
@maraffio72 Рік тому
Great video, thanks !
@danielheinisch7146
@danielheinisch7146 2 роки тому
Thanks for the informative video. In the end model in which uncertainty for different counties is very similar, isn´t the model understating this uncertainty for the cases with just 1 or 2 datapoints? Could you elaborate? Thanks!
@hhhtocode651
@hhhtocode651 Рік тому
Under Partial Pooling, why does sigma_a represent the degree of pooling?
@noejnava8409
@noejnava8409 2 роки тому
Omar, it is not clear to me why sigma controls the amount of pooling. Could you point me into some sources to learn more about this? I enjoyed your presentation. Thanks.
@williamchurcher9645
@williamchurcher9645 Рік тому
Hi Noé, maybe I can have a go at explaining. When doing the hierarchical modelling, we suppose that the parameters for each group themselves come from a distribution. If we assume that this distribution has zero variance, we are saying that all of the group-level parameters must be the same - they must be equal to the mean. This is because there is literally zero variance. This is the same as pooling all the data (the first example). On the other hand, if we assume the variance is very large, then each group parameter has the freedom to choose any value it wants, without penalisation from the group model-parameter distribution. This is the same as having no pooling - each group has its own parameter. We can choose sigma between these two extremes to specify how closely linked the group parameter should be. Thank you and I hope that helped!
@khanhtruong3254
@khanhtruong3254 Рік тому
Hi @@williamchurcher9645. Correct me if I'm wrong: given the prior distribution of alpha_i is assumed as Normal(mu_alpha, sigma_alpha), if sigma_alpha = 0, all the alpha_i may not be (and can not be) equal because the mu_alpha is not a fixed number but follows a distribution Normal(0, 5). Put that in a formula to be clearer: alpha_i ~ Normal(mu_alpha, sigma_alpha) alpha_i ~ Normal(mu_alpha, 0) alpha_i ~ Normal(Normal(0, 5), 0) alpha_i ~ Normal(0, 5) => alpha_i is not a constant but a distribution Following that understanding, the answer for why the sigma_alpha represents the degree of pooling is still vague.
Ах Ты Ж Су... Не Провоцируй Меня! @NutshellAnimations
00:15
Глеб Рандалайнен
Переглядів 3 млн
Stray Kids "Lose My Breath (Feat. Charlie Puth)" M/V
02:53
JYP Entertainment
Переглядів 11 млн
The Bayesians are Coming to Time Series
53:17
AICamp
Переглядів 21 тис.
Bayesian Hierarchical Models
49:19
NEON Science
Переглядів 13 тис.
Tech talk: Introduction to Bayesian modelling with PyStan
52:34
Introduction to Bayesian Statistics - A Beginner's Guide
1:18:47
Woody Lewenstein
Переглядів 71 тис.
Bayes theorem, the geometry of changing beliefs
15:11
3Blue1Brown
Переглядів 4,1 млн
Bayesian Methods in Modern Marketing Analytics with Juan Orduz
1:01:28