Multi-scale Information Diffusion Prediction with Reinforced Recurrent NetworksCheng Yang1,2,3∗ , Jian Tang4,5,6 , Maosong Sun1,2,3 , Ganqu Cui1,2,3 and Zhiyuan Liu1,2,3

1 Department of Computer Science and Technology, Tsinghua University, Beijing, China2 Institute for Artificial Intelligence, Tsinghua University, Beijing, China

3State Key Lab on Intelligent Technology and Systems, Tsinghua University, Beijing, China4Mila-Quebec Institute for Learning Algorithms, Canada

5HEC Montreal, Canada6Canadian Institute for Advanced Research (CIFAR)

AbstractInformation diffusion prediction is an importanttask which studies how information items spreadamong users. With the success of deep learningtechniques, recurrent neural networks (RNNs) haveshown their powerful capability in modeling infor-mation diffusion as sequential data. However, pre-vious works focused on either microscopic diffu-sion prediction which aims at guessing the nextinfluenced user or macroscopic diffusion predic-tion which estimates the total numbers of influ-enced users during the diffusion process. To thebest of our knowledge, no previous works havesuggested a unified model for both microscopicand macroscopic scales. In this paper, we pro-pose a novel multi-scale diffusion prediction modelbased on reinforcement learning (RL). RL incor-porates the macroscopic diffusion size informationinto the RNN-based microscopic diffusion modelby addressing the non-differentiable problem. Wealso employ an effective structural context extrac-tion strategy to utilize the underlying social graphinformation. Experimental results show that ourproposed model outperforms state-of-the-art base-line models on both microscopic and macroscopicdiffusion predictions on three real-world datasets.

1 IntroductionThe prediction of information diffusion, also known as cas-cade prediction, has been studied over a wide range of ap-plications, such as product adoption [Leskovec et al., 2007;Watts and Dodds, 2007; Aral and Walker, 2012], epidemi-ology [Wallinga and Teunis, 2004], social networks [Lappaset al., 2010; Dow et al., 2013] and the spread of news andopinions [Liben-Nowell and Kleinberg, 2008; Leskovec etal., 2009]. Recent works [Li et al., 2017; Cao et al., 2017;Islam et al., 2018; Wang et al., 2018b] on diffusion predic-tion took advantage of the success of deep learning tech-niques by modeling information diffusion as sequential data∗This work was partially done during the first au-

thor’s visit to MILA. Correspondence to: Cheng Yang<albertyang33@gmail.com>, Jian Tang <jian.tang@hec.ca>.

Figure 1: Illustrative examples for microscopic next infected userprediction (left) and macroscopic cascade size prediction (right).Conventionally, we usually use “infected” to indicate that a user is“influenced” by an information item.

based on recurrent neural networks (RNNs) and achievedpromising performances. Existing works [Wang et al., 2017a;Wang et al., 2018b] also explored the social graph informa-tion which is available when information diffusion spreadsthrough a social network service for diffusion prediction.

However, as shown in Fig. 1, previous works focused oneither microscopic diffusion prediction which aims at guess-ing the next infected user or macroscopic diffusion predictionwhich estimates the total numbers of infected users during thediffusion process. To the best of our knowledge, no previousworks have suggested a unified model for both microscopicand macroscopic scales. A unified model can utilize moreinformation in the training data especially for macroscopicdiffusion prediction, e.g. previous works [Li et al., 2017;Cao et al., 2017] considered cascade size prediction as a re-gression problem and discarded the information of detailedinfected users and the ordering of their infections.

Also, the handling of social graph information in existingworks [Wang et al., 2017a; Wang et al., 2018b] could be sub-optimal. [Wang et al., 2017a] only explored user pairs con-nected by direct social links and [Wang et al., 2018b] com-puted the similarities of all users and suffered from high timeand space complexity which is square to the number of users.

In this paper, we propose a novel multi-scale diffusion pre-diction model by endowing a microscopic cascade model theability to predict macroscopic cascade properties, i.e. the fi-nal size of a cascade, by a reinforcement learning (RL) frame-work. We further employ an efficient and effective structurecontext extraction method which was initially proposed forsemi-supervised graph classification [Hamilton et al., 2017]to utilize the social graph information. Experimental results

Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence (IJCAI-19)

4033

show that our proposed model achieves 10% and 12% rel-atively improvement against state-of-the-art baseline meth-ods on microscopic and macroscopic diffusion predictions,respectively.

To sum up, the contributions of this work are 3-fold:(1) We innovatively propose the problem of multi-scale dif-

fusion prediction by jointly considering the microscopic andmacroscopic diffusion prediction tasks.

(2) We propose a novel reinforcement learning frameworkto enable a microscopic cascade model for macroscopic dif-fusion predictions. We also employ a novel structural contextextraction algorithm to further take advantage of underlyingsocial graph information.

(3) Experimental results show that our proposed modeloutperforms state-of-the-art baseline methods on both micro-scopic and macroscopic diffusion predictions on three real-world datasets.

2 Related WorksOur work is related to microscopic and macroscopic diffusionprediction algorithms based on deep learning techniques. Wefurther group them into embedding-based and RNN-basedmethods according to their models. We summarize relatedwork in Table ??. “History” and “Graph” indicate whethera method uses the ordering or social graph of infected users.“Micro” and “Macro” are short for microscopic and macro-scopic tasks.

Method History Graph Micro Macro

Embedded IC XInf2vec X X

DeepCas X X XDeepHawkes X XCYAN-RNN X XTopoLSTM X X XDeepDiffuse X X

NDM X XSNIDSA X X Xthis work X X X X

Table 1: Summary of related works.

2.1 Embedding-based MethodsEmbedding-based methods target on microscopic level pre-dictions by extending IC-model [Kempe et al., 2003] whichassumed an independent diffusion probability for every userpair. [Bourigault et al., 2014; Gao et al., 2017] projectedusers into real-valued user embeddings and simplified the ICmodel by assuming that infected users are determined onlyby the source user. Embedded IC [Bourigault et al., 2016]modeled the diffusion probability between two users by afunction of their user embeddings instead of assigning a realnumber to each user pair. Inf2vec [Feng et al., 2018] fur-ther considered global user similarity context as an exten-sion. However, embedding-based methods failed to modelthe infection history, e.g. the ordering of infected users,for next infected user prediction and have been shown to

be suboptimal choices in the experiments of recent RNN-based approaches [Wang et al., 2017a; Wang et al., 2018b;Yang et al., 2018].

2.2 RNN-based MethodsFor macroscopic cascade models, DeepCas [Li et al., 2017]sampled sequences from social graph and observed cascades,and then employed RNN to encode the sequences and pre-dict the eventual size of a cascade. DeepHawkes [Cao et al.,2017] explored Hawkes self-exciting point process based onRNN architecture to utilize the infection timestamp informa-tion instead of social graph.

For microscopic cascade models, TopoLSTM [Wang etal., 2017a] extended the standard LSTM model by structur-ing the hidden states as a directed acyclic graph extractedfrom the social graph. CYAN-RNN [Wang et al., 2017b],DAN [Wang et al., 2018a] and DeepDiffuse [Islam et al.,2018] all employed RNN and attention model to utilize theinfection timestamp information. NDM [Yang et al., 2018]built a microscopic cascade model based on self-attention andconvolution neural networks motivated by two heuristic as-sumptions. SNIDSA [Wang et al., 2018b] computed pairwisesimilarities of all user pairs and incorporated the structural in-formation into RNN by a gating mechanism. However, to thebest of our knowledge, no previous works have suggested aunified model for both microscopic and macroscopic scales.

3 MethodIn this section, we will first formalize the diffusion predic-tion problem on both microscopic and macroscopic scales.Then we will propose a novel structural context extraction al-gorithm which was originally introduced for semi-supervisedgraph classification [Hamilton et al., 2017] to build an RNN-based microscopic cascade model. We further incorporate theability of macroscopic prediction, i.e. estimating the eventualsize of a cascade, into the model by reinforcement learning.Finally, we will present the overall algorithm and implemen-tation details.

3.1 Problem FormalizationGiven user set V , cascade set C, each cascade ci ∈ C isa sequence of users {vi1, vi2 . . . , vi|ci|} ranked by their infec-tion timestamps where |ci| is the size of cascade ci, i.e. thenumber of users infected by the corresponding item. In thispaper, we only keep the ordering of users and ignore the ex-act timestamps as previous works did [Wang et al., 2017a;Wang et al., 2018b; Yang et al., 2018] and leave the model-ing of timestamps as future works. Also, an underlying socialgraph G = (V,E) among users will be available when in-formation diffusion occurs on a social network service. Thesocial graph G will be considered as additional structural in-puts for diffusion prediction.

In this work, we target on both microscopic and macro-scopic diffusion predictions which focus on fine-grainedshort-term modeling and coarse-grained long-term estima-tion, respectively. We formalize the problems as below.

Microscopic Diffusion Prediction aims at predicting thenext infected user vik+1 given previously infected users

Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence (IJCAI-19)

4034

{vi1, vi2 . . . , vik} in cascade ci for k = 1, 2 . . . , |ci| − 1.Macroscopic Diffusion Prediction aims at predicting the

eventual size |ci| of cascade ci, i.e. the total number of in-fected users, given the first K infected users {vi1, vi2 . . . , viK}.

3.2 Microscopic Cascade ModelingIn this subsection, we will first present the Gated RecurrentUnit (GRU), a variant of RNNs, as the basis of microscopiccascade modeling. Then we introduce the structural contextextraction algorithm to utilize the social graph information.Afterward, we show how to combine these features for mi-croscopic diffusion prediction.

Recurrent NetworksRNNs have shown their effectiveness in sequential datamodeling in many areas, such as natural language process-ing [Mikolov et al., 2010]. Previous works [Wang et al.,2017a; Wang et al., 2018b; Islam et al., 2018] also ex-plored RNNs for cascade modeling and achieved promis-ing results. Specifically, we employ Gated Recurrent Unit(GRU) as the basis of our model. Given the cascade sequence{v1, v2 . . . , vk}, GRU takes user vt as input and computes ahidden state ht at each step t = 1, 2 . . . , k. We use Eq. 1 todenote the computation process of t-th step hidden state ht.

ht = GRU(ht−1, xvt), (1)

where xvt ∈ Rd is the d-dimensional embedding of user vt.The hidden state hk ∈ Rd encodes the history informa-

tion of all previously infected users {v1, v2 . . . , vk} in thecascade. Now we go on to encode the structural informationto take advantage of the underlying social graph.

Structural Context ExtractionFor each user v, we assume f

(0)v as its user features which

can be obtained from user profiles or pretrained node embed-dings. Our goal is to incorporate the structural informationinto the feature vector f

(0)v . Inspired by the recent success

in semi-supervised graph learning [Kipf and Welling, 2017;Hamilton et al., 2017], we employ an efficient structural con-text extraction algorithm based on neighborhood sampling forthe purpose.

Formally, we first sample Z users {u1, u2 . . . , uZ} from vand its neighbors N(v). Then we update the feature vectorf(0)v by aggregating the neighborhood features by Eq. 2.

f (1)v = relu(W · 1

Z

Z∑k=1

f (0)uk

+ b) (2)

where uk is uniformly sampled from user set {v} ∪N(v) fork = 1, 2 . . . Z, W, b are weight matrix and bias vector, andactivation function relu(·) = max(·, 0).

The updated user feature vector f (1)v encodes structural in-

formation by aggregating features from v’s first-order neigh-bors. As shown in Fig. 2, Eq. 2 can also be processed re-cursively to explore a larger neighborhood of user v. Empir-ically, a two-step neighborhood exploration is time-efficientand enough to give promising results. We will use fv to de-note the final user features for simplification.

Though initially proposed for semi-supervised graph clas-sification, we find the algorithm suitable for our problem.

Figure 2: An illustrative example of structural context extractionof the orange node by neighbor sampling and feature aggregation.(Best viewed in color)

Compared with previous works [Wang et al., 2018b] whichtakes O(|V |2) space and time complexity for structural con-text extraction, the neighborhood sampling and aggregationstrategy only take O(|V |) time and constant space complex-ity. Also, the algorithm can explore the two-step neighbor-hood besides directly connected neighbors.

Microscopic Diffusion PredictionNow we go back to the next infected user prediction prob-lem by combining the GRU and structural context. Givenpreviously infected users {vi1, vi2 . . . , vik} in cascade ci, thek-step hidden state hi

k of GRU encodes the sequential historyand user feature vectors fvi

1, fvi

2, . . . fvi

kencode the underly-

ing social graph information.Intuitively, the structural context of recent infected users

should contribute to the next infected user prediction becausethe diffusion may spread through the social links. Sincewe ignore the exact timestamps, we define “recent infectedusers” as the last m users {vik−m+1, v

ik−m+2 . . . , v

ik} where

m is a hyper-parameter controlling this window size. We fur-ther employ mean pooling1 to aggregate the feature vectorsas sik = mean(fvi

k−m+1, fvi

k−m+2, . . . fvi

k).

Finally, the probability of the next infected user is com-puted as

pik = softmax(Wp · concat(hik, s

ik) + bp), (3)

where pik ∈ R|V | is the multinomial probability distributionover all users, concat(·, ·) is the concatenation operation andWp, bp are weight matrix and bias vector, respectively.

The training objective of microscopic diffusion predictionis to maximize the log-likelihood of all cascades

Jmicro(Θ) =

|C|∑i=1

|ci|−1∑k=1

log pik[vik+1], (4)

where p[j] indicates the j-th dimension of vector p and Θdenotes all parameters in the microscopic cascade model.

1We also investigate other aggregation strategies such as atten-tion mechanism or concatenation. We find that other options willlead to model overfitting and suboptimal performances.

Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence (IJCAI-19)

4035

3.3 Macroscopic Cascade ModelingThe key of this work lies in how to endow a microscopic cas-cade model the ability to predict macroscopic cascade proper-ties, i.e. the size of a cascade. We divide our method into foursteps: (a) encode observed K users by a microscopic cascademodel; (b) enable the microscopic cascade model to predictthe size of a cascade by cascade simulations; (c) use Mean-Square Log-Transformed Error (MSLE) as the supervisionsignal for macroscopic predictions; and (d) employ a rein-forcement learning framework to update parameters throughpolicy gradient algorithm. The overall workflow is illustratedin Fig. 3.

Figure 3: The workflow of adopting microscopic cascade model formacroscopic size prediction by reinforcement learning.

Encoding Observed UsersWe feed observed K users of ci into the microscopic cas-cade model and get the last hidden state hi

K as shown in step(a) of Fig. 3. Also, we explicitly encode the positional in-formation which makes the model aware of how many usershave been inputted into GRU at each step. In specific, weassign a positional embedding POSt ∈ Rdpos for each stept = 1, 2 . . .maxlen where maxlen is the maximum lengthof cascades. At the t-th step of GRU in Eq. 1, we concatenatethe user embedding xvt and positional embedding POSt asthe input vector instead.

Cascade Simulation for Macroscopic PredictionTo adopt the microscopic cascade model for cascade size pre-diction in step (b), we firstly append a virtual user <STOP>to the end of every cascade and ask the model to predict itas well. To estimate the size of a cascade given the first Kinfected users, we recursively sample a user according to thepredicted probability distribution in Eq. 3, take it as the nextinput and make further predictions. Once the <STOP> sig-nal is predicted, we can count the users have been predictedalready as the final size of the cascade. Such cascade simu-lation will be processed for multiple times to reduce the vari-ance of estimations.

Supervision Signals for Macroscopic PredictionThough the modified microscopic cascade model can pre-dict the size of cascades by simulations, the model stillhas no supervision signals to guide the way towards bet-ter performances. In this paper, we employ Mean SquareLog-transformed Error (MSLE), which was used in previousworks [Li et al., 2017; Cao et al., 2017] as the evaluationmetric of cascade size prediction, for the supervision signal in

step (c) of Fig. 3: MSLE= 1|C|

∑|C|i=1(log(|ci|)−log(predi))

2

where predi is the predicted size of cascade ci.However, the sampling operation used in cascade size es-

timation is non-differentiable and makes it impossible to up-date the parameters by backward propagation. To overcomethis problem, we put the simulation process into the reinforce-ment learning (RL) framework and then employ policy gradi-ent to update the parameters as shown in step (d) of Fig. 3.

Policy Gradient for Parameter UpdatingWe map the GRU and its hidden state (including structuralcontext) to the agent and state concepts in RL. The actionat each step is to choose the next infected user and the pol-icy which decides the probability of actions given the currentstate is defined by Eq. 3. When the <STOP> action is cho-sen, a reward will be given as the opposite number of MSLE2.

Formally, for the size prediction of cascade ci, the first Kusers of ci are fed into the microscopic cascade model andthe last hidden state hi

K is used for the initial state of RL.For every action sequence seq = {a1, a2 . . . amaxlen} whereaj is the selected user in the j-th action, we can compute thereward reward(seq, ci) by the opposite number of MSLE.Then we aim to maximize the expectation of reward for cas-cade ci as

J iRL(Θ) =

∑seq

Pr(seq; Θ, hiK)reward(seq, ci), (5)

where Pr(seq; Θ, hiK) is the probability of choosing action

sequence seq and can be decomposed into the product of theprobability of each action aj . Note that seq is in the space of|V |maxlen and can not be enumerated to compute J i

RL. In-stead, the gradients of J i

RL can be computed by REINFORCEalgorithm [Williams, 1992]:

∇ΘJiRL =

∑seq

∇Θ Pr(seq; Θ, hiK) · reward(seq, ci)

=∑seq

Pr(seq; Θ, hiK)∇Θ log Pr(seq; Θ, hi

K) · reward(seq, ci)

= Eseq[∇Θ log Pr(seq; Θ, hiK) · reward(seq, ci)]

' 1

M

M∑m=1

∇Θ log Pr(seqm; Θ, hiK) · reward(seqm, ci),

(6)where seqm for m = 1, 2 . . .M are M samples fromPr(seq; Θ, hi

K) and the expectation over the whole action se-quence is approximated by Monte Carlo simulations at thelast step. Finally, parameters Θ will be updated by gradientascent to maximize the expectation of reward, i.e. the super-vision signal for macroscopic prediction.

3.4 Implementation DetailsFor the joint training of both microscopic and macroscopicobjectives, we first train the microscopic cascade model for10 epochs as warming up. The warming up phase will ensurethe next infected user prediction pik generates high-quality

2If the <STOP> action is not chosen when the maximum lengthmaxlen is reached, we assume this action sequence predicts the sizeas 2×maxlen.

Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence (IJCAI-19)

4036

Twitter Douban Memetracker

hits@10 hits@50 hits@1000

0.1

0.2

0.3

0.4

0.5

0.6hits

DeepDiffuse

TopoLSTM

NDM

SNIDSA

FOREST

hits@10 hits@50 hits@1000

0.05

0.1

0.15

0.2

0.25

0.3

0.35

hits

DeepDiffuse

TopoLSTM

NDM

SNIDSA

FOREST

hits@10 hits@50 hits@1000

0.1

0.2

0.3

0.4

0.5

0.6

hits

DeepDiffuse

NDM

FOREST

map@10 map@50 map@1000

0.05

0.1

0.15

0.2

map

map@10 map@50 map@1000

0.02

0.04

0.06

0.08

0.1

map

map@10 map@50 map@1000

0.05

0.1

0.15

0.2

map

Figure 4: Experimental results on microscopic diffusion prediction. For both MAP and HITS metrics, scores are the higher the better.

cascade simulations and help the RL training converge morequickly. Then we iteratively update the parameters to max-imize the microscopic and macroscopic objectives until wereach the best performance on validation data. We employAdam [Kingma and Ba, 2015] optimizer for gradient ascent.

For hyper-parameter settings, the dimension of hidden stateand user feature vector d = 64, controlling window size m =3, neighbors sampled in structural context extraction Z1 =25, Z2 = 10 for first-order and second-order aggregation, thedimension of positional embedding dpos = 8 and trainingdata are grouped into mini-batches with batch size 16.

We name our method as reinFOrced REcurrent networkswith STructural context (FOREST). The source code ofthis paper can be found at https://github.com/albertyang33/FOREST.

4 ExperimentsWe conduct experiments on both microscopic and macro-scopic cascade predictions to demonstrate the effectivenessof our proposed model.

4.1 DatasetsTwitter [Hodas and Lerman, 2014] dataset records the tweetscontaining URLs during October 2010. Each URL is inter-preted as an information item spreading among users.

Douban [Zhong et al., 2012] is a Chinese social websitewhere users can update their book reading statuses and followthe statuses of other users. Each book is considered as aninformation item and a user is infected if she reads the book.

Memetracker [Leskovec et al., 2009] collects a million ofnews stories and blog posts from online websites and track themost frequent quotes and phrases, i.e. memes, to analyze the

Dataset # Users # Links # Cascades Avg. Length

Twitter 12,627 309,631 3,442 32.60Douban 23,123 348,280 10,602 27.14

Memetracker 4,709 - 12,661 16.24

Table 2: Statistics of datasets.

migration of memes among people. Each meme is regardedas an information item and each URL of websites is treated asa user. Note that this dataset has no underlying social graph.

We randomly sample 80% of cascades for training, 10% forvalidation and the rest 10% for test. The statistics of datasetsare listed in Table 2.

4.2 BaselinesFor a thorough comparison, we consider five very recentbaseline methods on both microscopic and macroscopic cas-cade predictions. Microscopic cascade prediction models:

TopoLSTM [Wang et al., 2017a] extends the standardLSTM model by structuring the hidden states as a directedacyclic graph which is extracted from the social graph.

DeepDiffuse [Islam et al., 2018] employs embedding tech-nique and attention model to utilize the infection timestampinformation. We replace the timestamps by infection steps aswe ignore the exact timestamps in the datasets.

NDM [Yang et al., 2018] builds a microscopic cascademodel based on self-attention and convolution neural net-works to alleviate the long-term dependency problem.

SNIDSA [Wang et al., 2018b] computes pairwise similar-ities of all user pairs and incorporates the structural informa-tion into RNN by a gating mechanism.

Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence (IJCAI-19)

4037

Twitter Douban Memetracker

0

0.5

1

1.5

2

2.5M

SLE

DeepCas

DeepDiffuse

NDM

SNIDSA

FOREST-RL

FOREST

0

1

2

3

4

5

MS

LE

DeepCas

DeepDiffuse

NDM

SNIDSA

FOREST-RL

FOREST

0

0.5

1

1.5

MS

LE

DeepCas

DeepDiffuse

NDM

FOREST-RL

FOREST

Figure 5: Experimental results on macroscopic diffusion prediction. The scores are the lower the better. FOREST-RL is the variant ofFOREST by removing the RL part. We omit the results of TopoLSTM because its scores are larger than 10 and cannot be fit into the figure.

Macroscopic cascade prediction models:DeepCas [Li et al., 2017] is the state-of-the-art cascade

size prediction algorithm which considers both cascade in-formation and underlying social graph.

4.3 Experimental SettingsFor microscopic prediction, we consider the next infecteduser prediction as a retrieval task by ranking the uninfectedusers by their infection probabilities in Eq. 3. We report theMean Average Precision (MAP) and HITS scores. The samesettings are used in [Wang et al., 2017a; Islam et al., 2018].

For macroscopic prediction, the first K = 5 users in acascade are given to predict the size of a cascade. We useMSLE which is detailed in section 3.3 for evaluation. Thesame evaluation metric is used in [Li et al., 2017]. Also, allmicroscopic baselines are adopted for macroscopic cascadesize prediction by appending an additional <STOP> signalto the training cascades.

For Twitter and Douban datasets, we use pretrained Deep-Walk [Perozzi et al., 2014] embedding with dimension d =

64 as initial user feature vectors f(0)v . We excludes TopoL-

STM and SNIDSA for Memetracker because of the absenceof underlying social graph.

4.4 Experimental ResultsFigure 4 and 5 present the experimental results on micro-scopic and macroscopic diffusion predictions, respectively.We have the following observations:

(1) FOREST consistently outperforms all state-of-the-artbaseline methods on microscopic diffusion prediction by arelative improvement of more than 10% in terms of HITSand MAP scores. Compared with TopoLSTM and SNIDSA,the improvements mostly come from the encoding of struc-tural context. The structural context extraction algorithm ofFOREST considers second-order neighborhoods while previ-ous works only considered the first-order neighbors.

(2) FOREST consistently outperforms other baselines in-cluding DeepCas, the state-of-the-art macroscopic diffusionprediction method, on cascade size prediction by a relativeimprovement of more than 12% in terms of MSLE. Com-pared with FOREST-RL where the reinforcement training ofmacroscopic size prediction is removed, FOREST achieves

more promising and robust performances by incorporatingmacroscopic supervision signals for parameter training.

(3) Compared with DeepCas, microscopic cascade modelsare able to take advantage of more information in the trainingdata, i.e. the detailed infected users and the ordering of theirinfections. Thus microscopic cascade models are able to givecomparable and even better results on macroscopic diffusionprediction task. This finding would encourage microscopiccascade models to replace macroscopic ones in future works.

(4) We omit the ablation study of FOREST-RL and FOR-EST in microscopic prediction because the benefit from RLfor microscopic prediction is not as significant as that formacroscopic prediction (30%-80% relative improvement inFig. 5). This is because reinforcement learning focuseson long-term modeling and microscopic prediction (next in-fected user prediction) is short-term. We will consider a long-term microscopic prediction setting for future work.

5 ConclusionIn this paper, we propose a novel multi-scale diffusion predic-tion model, FOREST, for both microscopic and macroscopicpredictions. In specific, we adopt microscopic cascade mod-els for macroscopic cascade size prediction by a reinforce-ment learning framework which incorporates macroscopicsupervision and achieves promising performances. Also, weemploy an effective structural context extraction method toutilize the underlying social graph information. Experimen-tal results on next infected user and cascade size predictionsdemonstrate the effectiveness of our method.

For future works, an intriguing direction is to consider theexact timestamp information which is ignored in this work formodeling. We will also investigate the possibility of applyingthe RL module on other microscopic cascade models.

AcknowledgmentsThis research is jointly supported by the NSFC project un-der the grant no. 61661146007 and the NExT++ project,the National Research Foundation, Prime Minister’s Office,Singapore under its IRC@Singapore Funding Initiative. JianTang is supported by the Natural Sciences and EngineeringResearch Council of Canada, as well as the Canada CIFARAI Chair Program.

Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence (IJCAI-19)

4038

References[Aral and Walker, 2012] Sinan Aral and Dylan Walker. Iden-

tifying influential and susceptible members of social net-works. Science, 2012.

[Bourigault et al., 2014] Simon Bourigault, Cedric Lagnier,Sylvain Lamprier, Ludovic Denoyer, and Patrick Gallinari.Learning social network embeddings for predicting infor-mation diffusion. In Proceedings of WSDM. ACM, 2014.

[Bourigault et al., 2016] Simon Bourigault, Sylvain Lam-prier, and Patrick Gallinari. Representation learning for in-formation diffusion through social networks: an embeddedcascade model. In Proceedings of WSDM. ACM, 2016.

[Cao et al., 2017] Qi Cao, Huawei Shen, Keting Cen, Wen-tao Ouyang, and Xueqi Cheng. Deephawkes: Bridging thegap between prediction and understanding of informationcascades. In Proceedings of CIKM. ACM, 2017.

[Dow et al., 2013] P Alex Dow, Lada A Adamic, and AdrienFriggeri. The anatomy of large facebook cascades.ICWSM, 2013.

[Feng et al., 2018] Shanshan Feng, Gao Cong, Arijit Khan,Xiucheng Li, Yong Liu, and Yeow Meng Chee. Inf2vec:Latent representation model for social influence embed-ding. In Proceedings of ICDE, 2018.

[Gao et al., 2017] Sheng Gao, Huacan Pang, Patrick Galli-nari, Jun Guo, and Nei Kato. A novel embedding methodfor information diffusion prediction in social network bigdata. IEEE Transactions on Industrial Informatics, 2017.

[Hamilton et al., 2017] Will Hamilton, Zhitao Ying, and JureLeskovec. Inductive representation learning on largegraphs. In Proceedings of NeurIPS, 2017.

[Hodas and Lerman, 2014] Nathan O Hodas and KristinaLerman. The simple rules of social contagion. Scientificreports, 4:4343, 2014.

[Islam et al., 2018] Mohammad Raihanul Islam, Sathap-pan Muthiah, Bijaya Adhikari, B Aditya Prakash,and Naren Ramakrishnan. Deepdiffuse: Predictingthe’who’and’when’in cascades. In Proceedings of ICDM,2018.

[Kempe et al., 2003] David Kempe, Jon Kleinberg, and EvaTardos. Maximizing the spread of influence through a so-cial network. In Proceedings of SIGKDD, 2003.

[Kingma and Ba, 2015] Diederik Kingma and Jimmy Ba.Adam: A method for stochastic optimization. In Proceed-ings of ICLR, 2015.

[Kipf and Welling, 2017] Thomas N. Kipf and Max Welling.Semi-supervised classification with graph convolutionalnetworks. In Proceedings of ICLR, 2017.

[Lappas et al., 2010] Theodoros Lappas, Evimaria Terzi,Dimitrios Gunopulos, and Heikki Mannila. Finding effec-tors in social networks. In Proceedings of SIGKDD, pages1059–1068. ACM, 2010.

[Leskovec et al., 2007] Jure Leskovec, Lada A Adamic, andBernardo A Huberman. The dynamics of viral marketing.ACM Transactions on the Web (TWEB), 1(1):5, 2007.

[Leskovec et al., 2009] Jure Leskovec, Lars Backstrom, andJon Kleinberg. Meme-tracking and the dynamics of thenews cycle. In Proceedings of SIGKDD. ACM, 2009.

[Li et al., 2017] Cheng Li, Jiaqi Ma, Xiaoxiao Guo, andQiaozhu Mei. Deepcas: An end-to-end predictor of in-formation cascades. In Proceedings of WWW, pages 577–586. International World Wide Web Conferences SteeringCommittee, 2017.

[Liben-Nowell and Kleinberg, 2008] David Liben-Nowelland Jon Kleinberg. Tracing information flow on a globalscale using internet chain-letter data. Proceedings of thenational academy of sciences, 105(12):4633–4638, 2008.

[Mikolov et al., 2010] Tomas Mikolov, Martin Karafiat,Lukas Burget, Jan Cernocky, and Sanjeev Khudanpur. Re-current neural network based language model. In Eleventhannual conference of the international speech communi-cation association, 2010.

[Perozzi et al., 2014] Bryan Perozzi, Rami Al-Rfou, andSteven Skiena. Deepwalk: Online learning of social rep-resentations. In Proceedings of SIGKDD, pages 701–710.ACM, 2014.

[Wallinga and Teunis, 2004] Jacco Wallinga and Peter Teu-nis. Different epidemic curves for severe acute respira-tory syndrome reveal similar impacts of control measures.American Journal of epidemiology, 2004.

[Wang et al., 2017a] Jia Wang, Vincent W Zheng, ZeminLiu, and Kevin Chen-Chuan Chang. Topological recurrentneural network for diffusion prediction. In ICDM, pages475–484. IEEE, 2017.

[Wang et al., 2017b] Yongqing Wang, Huawei Shen,Shenghua Liu, Jinhua Gao, and Xueqi Cheng. Cascadedynamics modeling with attention-based recurrent neuralnetwork. In Proceedings of IJCAI, 2017.

[Wang et al., 2018a] Zhitao Wang, Chengyao Chen, andWenjie Li. Attention network for information diffusionprediction. In Proceedings of WWW, 2018.

[Wang et al., 2018b] Zhitao Wang, Chengyao Chen, andWenjie Li. A sequential neural information diffusionmodel with structure attention. In Proceedings of CIKM,2018.

[Watts and Dodds, 2007] Duncan J Watts and Peter SheridanDodds. Influentials, networks, and public opinion forma-tion. Journal of consumer research, 34(4):441–458, 2007.

[Williams, 1992] Ronald J Williams. Simple statisticalgradient-following algorithms for connectionist reinforce-ment learning. Machine learning, 8(3-4):229–256, 1992.

[Yang et al., 2018] Cheng Yang, Maosong Sun, Haoran Liu,Shiyi Han, Zhiyuan Liu, and Huanbo Luan. Neural dif-fusion model for microscopic cascade prediction. arXivpreprint arXiv:1812.08933, 2018.

[Zhong et al., 2012] Erheng Zhong, Wei Fan, Junwei Wang,Lei Xiao, and Yong Li. Comsoc: Adaptive transfer of userbehaviors over composite social network. In Proceedingsof SIGKDD, 2012.

Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence (IJCAI-19)

4039