L1, L4-L9
EU Data Act | Article 2, Definitions:
👍 Defines the target study/register population
👎 Define exceptions to the general rules
It is widely acknowledged that the most fundamental developments in statistics in the past 60 years are driven by information technology (IT). We should not underestimate the importance of pen and paper as a form of IT but it is since people start using computers to do statistical analysis that we really changed the role statistics plays in our research as well as normal life.
Although: “Let’s not kid ourselves: the most widely used piece of software for statistics is Excel.”” /Brian Ripley (2002)
Design principles:
Modern usage includes:
Git is integrated into:
Today, version control supports:
.gitignoreCommon file structures
An article about computational science in a scientific publication is not the scholarship itself, it is merely advertising of the scholarship. The actual scholarship is the complete software development environment and the complete set of instructions which generated the figures. / Buckheit & Donoho
[It is important to have] a clear understanding of how data analysis was conducted when critical life and death decisions must be made. / Peng et al. 2021
A published data analysis is reproducible if the analytic data sets and the computer code used to create the data analysis are made available to others for independent study and analysis
R should behave similar on different operating systems.
Exceptions nevertheless exist!
Different names, same concept:
Connect through VPN
2-factor authentication (BankID common in Sweden)
Remote Desktop
Limited internet access from within the environment (maybe possible to install CRAN-packages tied to a certain historical snapshot but not the latest released/developed versions).
Data import and exports goes through administrator
Pros: Secure, centralized backups, possibility to scale/share large computing resources (CPU, RAM, GPU etc), less dependent on individual computer (laptop), might aid collaboration
Cons: needs internet connection, restrictions on what software (packages) to use, difficult to export results and/or import script files etc, mentally exhausting to use for example a Mac Keyboard when working in Windows, can be expensive (pay per clock-cycle of CPU usage etc instead of simply purchasing a computer once).
haven::read_sas()a database is a collection of “tables” (tabular data frames)
Differences between data frames and database tables:
data.tables do, which is one of the reasons that they’re so fast.Most classical databases are optimized for rapidly collecting data, not analyzing existing data.
These databases are called row-oriented because the data is stored row-by-row, rather than column-by-column like R.
More recently, there’s been much development of column-oriented databases that make analyzing the existing data much faster.
This query:
DuckDB can query files directly:
No data import (to your computers Random Access Memory [RAM]) is required!
{duckplyr} makes it possible to work with data which does not fit into memory{data.table} is much more efficient!data.frame in base R or tibble in the tidyverse{pxweb} R package simplifies the processICD has evolved from a mortality list to a comprehensive disease classification.
Several countries use national adaptations:
An ICD code reflects:
When analyzing long time series:
Crosswalks are rarely exact. Information loss or ambiguity is common.
Aggregation to broader diagnostic groups is often necessary.
In Sweden, medical procedures are coded using the NOMESCO Classification of Surgical Procedures (NCSP).
SNOMED CT (Systematized Nomenclature of Medicine – Clinical Terms) is a large clinical terminology system.
Unlike ICD or ATC, SNOMED CT is primarily a terminology, not a statistical classification.
| System | Type | Purpose |
|---|---|---|
| ICD | Classification | Epidemiology and health statistics |
| ATC | Classification | Drug classification |
| KVÅ / NOMESCO | Classification | Medical procedures |
| SNOMED CT | Terminology | Detailed clinical documentation |
Classification systems simplify reality for statistics and reporting, while terminologies allow very detailed clinical descriptions.
During childhood and school years, health information is collected through:
In contrast to many other stages of life in Sweden, these data are not collected in a national register.
The conscription register contains information from military conscription examination
These programmes aim to detect disease at an early stage in otherwise healthy individuals.
Data from screening programmes are often stored in regional systems
Primary care accounts for a large share of health care contacts in Sweden, but there is still no comprehensive national primary care register comparable to NPR.
Dispension does not guarantee that the patient actually used the medication
Not a health care register but often used for background data.
National quality registers collect detailed clinical information on specific diseases or treatments.
They are designed to:
But they can also be used for research
Typical characteristics:
There are currently around 100 national quality registers in Sweden.
Key characteristics:
Limitations:
The Nordic countries have relatively similar systems based on:
These countries are therefore often used in comparative register-based research.
Health data systems in other countries are often organised differently:
These systems can provide valuable data, but often lack:
# clean data and merge cohorts)# TODO:)README.mdModern biostatistics projects rarely involve only statistics.
A good workflow should make it possible to combine all of these steps in one place.
Quarto is a publishing system for scientific and technical documents.
It allows you to combine:
in a single workflow.
This makes it easier to create reproducible statistical reports.
code-fold.
.docx) etc