Learning Bayesian Networks from continuous data is an challanging task. In bnlearn this task is now accomplished by learning discrete bayesian networks from continuous data.

In order to do this, I am using a Bayesian discretization method for continuous variables in Bayesian networks with quadratic complexity instead of the cubic complexity of other standard techniques. Empirical demonstrations show that the proposed method is superior to the established minimum description length algorithm. The method is described in the paper of Yi-Chun Chen et al.

The underlying idea i that the implemented method now can perform structure learning by first discretizng the continuous variables and simultaneously learn Bayesian network structures.

Advanced discretizing continous data

To demonstrate the usage of parameter learning on continuous data, I will use the well known auto mpg data set.

# Import
import bnlearn as bn

# Load data set
df = bn.import_example(data='auto_mpg')
# Print
print(df)

#       mpg  cylinders  displacement  ...  acceleration  model_year  origin
# 0    18.0          8         307.0  ...          12.0          70       1
# 1    15.0          8         350.0  ...          11.5          70       1
# 2    18.0          8         318.0  ...          11.0          70       1
# 3    16.0          8         304.0  ...          12.0          70       1
# 4    17.0          8         302.0  ...          10.5          70       1
# ..    ...        ...           ...  ...           ...         ...     ...
# 387  27.0          4         140.0  ...          15.6          82       1
# 388  44.0          4          97.0  ...          24.6          82       2
# 389  32.0          4         135.0  ...          11.6          82       1
# 390  28.0          4         120.0  ...          18.6          82       1
# 391  31.0          4         119.0  ...          19.4          82       1
#
# [392 rows x 8 columns]

# Define the edges
edges = [
    ("cylinders", "displacement"),
    ("displacement", "model_year"),
    ("displacement", "weight"),
    ("displacement", "horsepower"),
    ("weight", "model_year"),
    ("weight", "mpg"),
    ("horsepower", "acceleration"),
    ("mpg", "model_year"),
]

# Create DAG based on edges
DAG = bn.make_DAG(edges)

We can now discretize the continuous columns as following:

# A good habbit is to set the columns with continuous data as float
continuous_columns = ["mpg", "displacement", "horsepower", "weight", "acceleration"]

# Discretize the continous columns by specifying
df_discrete = bn.discretize(df, edges, continuous_columns, max_iterations=1)

#                 mpg  cylinders  ... model_year origin
# 0     (17.65, 21.3]          8  ...         70      1
# 1    (8.624, 15.25]          8  ...         70      1
# 2     (17.65, 21.3]          8  ...         70      1
# 3    (15.25, 17.65]          8  ...         70      1
# 4    (15.25, 17.65]          8  ...         70      1
# ..              ...        ...  ...        ...    ...
# 387   (25.65, 28.9]          4  ...         82      1
# 388    (28.9, 46.6]          4  ...         82      2
# 389    (28.9, 46.6]          4  ...         82      1
# 390   (25.65, 28.9]          4  ...         82      1
# 391    (28.9, 46.6]          4  ...         82      1
#
# [392 rows x 8 columns]

At this point it is not different than any other discrete data set. We can specify the DAG together with the discrete data frame and fit a model using bnlearn.

Structure learning

We will learn the structure on the continuous data. Note that the data is also discretezed on a set of edges which will likely introduce a bias in the learned structure.

# Learn the structure
model = bn.structure_learning.fit(df_discrete, methodtype='hc', scoretype='bic')

# Independence test
model = bn.independence_test(model, df, prune=True)
# [bnlearn] >Compute edge strength with [chi_square]
# [bnlearn] >Edge [weight <-> mpg] [P=0.999112] is excluded because it was not significant (P<0.05) with [chi_square]

# Make plot
bn.plot(model)
# Create interactive plot
bn.plot(model, interactive=True)

Parameter learning

Let’s continue with parameter learning on the continuous data set and see whether we can estimate the CPDs.

# Fit model based on DAG and discretized continous columns
model = bn.parameter_learning.fit(DAG, df_discrete)

# Use MLE method
# model_mle = bn.parameter_learning.fit(DAG, df_discrete, methodtype="maximumlikelihood")

After fitting the model on the DAG and data frame, we can perform the independence test to remove any spurious edges and create a plot. In this case, the tooltips will contain the CPDs as these are computed with parameter learning.

# Independence test
model = bn.independence_test(model, df, prune=True)

# Make plot
bn.plot(model)
# Create interactive plot.
bn.plot(model, interactive=True)

There are various manners to deeper investigate the results such as looking at the CPDs.

# Print CPDs
print(model["model"].get_cpds("mpg"))

weight	…	weight((3657.5, 5140.0])
mpg((8.624, 15.25])	…	0.29931972789115646
mpg((15.25, 17.65])	…	0.19727891156462582
mpg((17.65, 21.3])	…	0.13313896987366375
mpg((21.3, 25.65])	…	0.12439261418853255
mpg((25.65, 28.9])	…	0.12439261418853255
mpg((28.9, 46.6])	…	0.12147716229348882

print("Weight categories: ", df_disc["weight"].dtype.categories)
# Weight categories:  IntervalIndex([(1577.73, 2217.0], (2217.0, 2959.5], (2959.5, 3657.5], (3657.5, 5140.0]], dtype='interval[float64, right]')

Inference

Making inferences can be perfomred using the fitted model. Note that the evidence should be discretized for which we can use the discretize_value function.

evidence = {"weight": bn.discretize_value(df_discrete["weight"], 3000.0)}
print(evidence)
# {'weight': Interval(2959.5, 3657.5, closed='right')}

print(bn.inference.fit(model, variables=["mpg"], evidence=evidence, verbose=0))

mpg	phi(mpg)
mpg((8.624, 15.25])	0.1510
mpg((15.25, 17.65])	0.1601
mpg((17.65, 21.3])	0.2665
mpg((21.3, 25.65])	0.1540
mpg((25.65, 28.9])	0.1327
mpg((28.9, 46.6])	0.1358

References

1. Yi-Chun Chen, Tim Allan Wheeler, Mykel John Kochenderfer (2015), Learning Discrete Bayesian Networks from Continuous Data arxiv 1512.02406

2. Julia 0.4 implementation: https://github.com/sisl/LearnDiscreteBayesNets.jl

---

---