IEEE CS Literature Review Example: GNNs for Code Representation

All citations verified against arXiv, ACM DL, NeurIPS/ICLR proceedings, and IEEE Xplore. No effect sizes (survey form); no fabricated DOIs.

Survey body (IEEE numeric-citation form)

1. Motivation and Scope

Learning distributed representations of source code shifted from token-sequence models to structure-aware models once it became clear that syntax and data/control flow carry information not recoverable from flattened text [1]. Early structural work used Tree-LSTM, a recursive encoder later adapted to code ASTs [2]—Tai, Socher, and Manning introduced it for natural-language constituency trees, and the architecture was subsequently transferred to AST encoding. But tree recursion cannot model cross-tree edges such as data dependencies or call relations. Graph neural networks close that gap by treating code as a heterogeneous graph where nodes are AST nodes, variables, or statements, and edges encode syntactic parentage, control flow, data flow, and token succession [3].

1. Graph Construction from Source

Three construction families dominate the literature.

(a) AST-based graphs. The AST is the default backbone. code2vec [4] and code2seq [5] do not run a GNN but establish the AST-path abstraction that later GNN models inherit: a code snippet is a bag of paths between terminal AST nodes, each path encoded and attention-aggregated. Allamanis, Brockschmidt, and Khademi [3] extend this into a true graph by adding syntactic edges plus "next-token" and "last-use" edges, then run a Gated Graph Neural Network (GGNN) to predict variable names and method names. This paper is the bridge from path-based embedding to message passing on code graphs.

(b) Control/data-flow augmentation. Pure ASTs miss execution semantics. Devign [6] builds a heterogeneous AST+CFG+DFG+PDG graph and applies a GNN with a convolution-style aggregator for vulnerability identification; it remains the standard baseline for vulnerability-detection surveys. Hoppity [7] augments the AST with SuccToken edges and ValueLink edges, then learns graph transformations for automated bug fixing.

(c) Heterogeneous and multi-view graphs. Later work (e.g., GREAT [8], GraphCodeBERT [9]) treats code as multiple aligned views (token sequence + AST + data flow) and either fuses them pre-GNN or uses graph-guided attention over the transformer. Zhang et al. [10] learn heterogeneous program graphs combining AST, CFG, and DFG in one message-passing pass.

1. GNN Operators Used on Code

The operator choice tracks the general GNN literature [11]: • GGNN [3][6] — gated recurrence over fixed steps; dominant in early code-GNN work because it handles edge types via typed adjacency.

• GCN [12] — used when the code graph is homogenized (e.g., Devign's combined graph convolved without edge-type gates).

• GAT / edge-typed attention [8][9] — replaces typed adjacency with learned attention per edge type; better on heterogeneous graphs.

• Message Passing (MPNN) — the unifying framework; most 2020+ code-GNN papers describe their model as an MPNN with a custom edge-merge function [11].

1. Task Grounding

Task Representative GNN work Graph type

Method/var name prediction Allamanis et al. 2018 [3] AST + next-token/last-use

Vulnerability detection Devign [6] AST+CFG+DFG+PDG

Bug localization / fix Hoppity [7] AST + SuccToken/ValueLink

Code search / clone detection GraphCodeBERT [9], Zhang et al. [10] Heterogeneous multi-view

Type inference LambdaNet [13] AST + let-binding edges

1. Open Problems

Four gaps recur across surveys [1][11][12]: (i) scalability to whole-repository graphs (most models truncate to function level); (ii) generalization across languages when edge semantics differ (e.g., Rust borrow edges have no Java equivalent); (iii) interpretability of what GNN attention actually attends to in a CFG; (iv) pretraining objectives—whether masked-node or edge-masking transfers better than sequence MLM when fused with transformers [9].

Numeric-citation audit (IEEE)

[n] Citation First appearance

[1] Allamanis, M., Barr, E., Devanbu, P., Sutton, C. "A Survey of Machine Learning for Big Code and Naturalness." ACM Comput. Surv., 51(4), 2018 §1

[2] Tai, K. S., Socher, R., Manning, C. D. "Improved Semantic Representations From Tree-Structured LSTM." ACL 2015, P15-1150, pp. 1556–1566 §1

[3] Allamanis, M., Brockschmidt, M., Khademi, M. "Learning to Represent Programs with Graphs." ICLR 2018, arXiv:1711.00740 §2(a)

[4] Alon, U., Zilberstein, M., Levy, O., Yahav, E. "code2vec: Learning Distributed Representations of Code." POPL 2019 §2(a)

[5] Alon, U., Brody, S., Levy, O., Yahav, E. "code2seq: Generating Sequences from Structured Representations of Code." ICLR 2019 §2(a)

[6] Zhou, Y., Liu, S., Siow, J., Du, X., Liu, Y. "Devign: Effective Vulnerability Identification by Learning Comprehensive Program Semantics via GNNs." NeurIPS 2019, 32, pp. 10197–10207 §2(b)

[7] Dinella, E., Dai, H., Li, Z., Naik, M., Song, L., Wang, K. "Hoppity: Learning Graph Transformations to Detect and Fix Bugs in Programs." ICLR 2020 §2(b)

[8] Hellendoorn, V. J., Sutton, C., Singh, R., Maniatis, P., Bieber, D. "Global Relational Models of Source Code." ICLR 2020 §3

[9] Guo, D., Ren, S., Lu, S., Feng, Z., Tang, D., Liu, S., Zhou, L., Duan, N., Svyatkovskiy, A., Fu, S., Tufano, M., Deng, S., Clement, C., Drain, D., Sundaresan, N., Yin, J., Jiang, D., Zhou, M. "GraphCodeBERT: Pre-training Code Representations with Data Flow." ICLR 2021 §2(c)

[10] Zhang, K., Wang, W., Zhang, H., et al. "Learning to Represent Programs with Heterogeneous Graphs." ICPC 2022 §2(c)

[11] Zhou, J., Cui, G., Hu, S., Zhang, Z., Yang, C., Liu, Z., Wang, L., Li, C., Sun, M. "Graph Neural Networks: A Review of Methods and Applications." AI Open, 1, pp. 57–81, 2020, arXiv:1812.08434 §3

[12] Kipf, T. N., Welling, M. "Semi-Supervised Classification with Graph Convolutional Networks." arXiv:1609.02907, 2017 (originally presented at ICLR 2017 Workshop on Representation Learning on Graphs and Manifolds) §3

[13] Wei, J., Goyal, M., Durrett, G., Dillig, I. "LambdaNet: Probabilistic Type Inference Using GNNs." ICLR 2020 §4

Reading-order rule: numbers appear in the order references are first cited, not alphabetically. If a survey inserts a new citation mid-text, all later numbers shift—IEEE requires renumbering, unlike Harvard/APA.

[12] provenance note: GCN's primary reference is the arXiv:1609.02907 technical report (Kipf & Welling, Univ. of Amsterdam, 2017). It was also presented at the ICLR 2017 Workshop on Representation Learning on Graphs and Manifolds—but it is not an ICLR main-track paper, so it must not be cited as "in Proc. ICLR."

References (IEEE format)

[1] M. Allamanis, E. Barr, P. Devanbu, and C. Sutton, "A survey of machine learning for big code and naturalness," ACM Comput. Surv., vol. 51, no. 4, pp. 1–37, 2018, doi:10.1145/3212695.

[2] K. S. Tai, R. Socher, and C. D. Manning, "Improved semantic representations from tree-structured long short-term memory networks," in Proc. ACL, 2015, pp. 1556–1566, doi:10.18653/v1/P15-1150.

[3] M. Allamanis, M. Brockschmidt, and M. Khademi, "Learning to represent programs with graphs," in Proc. ICLR, 2018, arXiv:1711.00740.

[4] U. Alon, M. Zilberstein, O. Levy, and E. Yahav, "code2vec: Learning distributed representations of code," Proc. ACM Program. Lang., vol. 3, no. POPL, Article 40, 2019, doi:10.1145/3290353.

[5] U. Alon, S. Brody, O. Levy, and E. Yahav, "code2seq: Generating sequences from structured representations of code," in Proc. ICLR, 2019.

[6] Y. Zhou, S. Liu, J. Siow, X. Du, and Y. Liu, "Devign: Effective vulnerability identification by learning comprehensive program semantics via graph neural networks," in Adv. Neural Inf. Process. Syst. (NeurIPS), vol. 32, 2019, pp. 10197–10207.

[7] E. Dinella, H. Dai, Z. Li, M. Naik, L. Song, and K. Wang, "Hoppity: Learning graph transformations to detect and fix bugs in programs," in Proc. ICLR, 2020.

[8] V. J. Hellendoorn, C. Sutton, R. Singh, P. Maniatis, and D. Bieber, "Global relational models of source code," in Proc. ICLR, 2020.

[9] D. Guo et al., "GraphCodeBERT: Pre-training code representations with data flow," in Proc. ICLR, 2021.

[10] K. Zhang, W. Wang, H. Zhang, et al., "Learning to represent programs with heterogeneous graphs," in Proc. ICPC, 2022.

[11] J. Zhou, G. Cui, S. Hu, Z. Zhang, C. Yang, Z. Liu, L. Wang, C. Li, and M. Sun, "Graph neural networks: A review of methods and applications," AI Open, vol. 1, pp. 57–81, 2020, doi:10.1016/j.aiopen.2021.01.001.

[12] T. N. Kipf and M. Welling, "Semi-supervised classification with graph convolutional networks," arXiv:1609.02907, 2017. [Online]. Available: https://arxiv.org/abs/1609.02907 (originally presented at ICLR 2017 Workshop on Representation Learning on Graphs and Manifolds).

[13] J. Wei, M. Goyal, G. Durrett, and I. Dillig, "LambdaNet: Probabilistic type inference using graph neural networks," in Proc. ICLR, 2020.

IEEE format checks: • References (not "Bibliography"), items numbered [1]–[13]

• Author list: initials first (M. Allamanis); up to 6 authors all listed, 7+ use et al. (IEEE-2023; [9] lists 6 then et al.—acceptable under IEEE-2018/2023)

• Article title in double quotes, journal/book italicized

• vol., no., pp., month-year or year-only per venue

• Conference: in Proc. ICPC, 2022 (no location required in current IEEE)

• arXiv as arXiv:XXXX.XXXXX at end of entry; [12] uses [Online]. Available: https://arxiv.org/abs/...

• DOI as doi:10.xxxx/...

Poor → Strong (survey form)

Element Poor IEEE survey Strong IEEE survey (above)

Citation style (Allamanis et al., 2018) [3] numeric, reading order

Reference list Alphabetical "References" Numbered [1]–[13], initials-first

Taxonomy "GNNs are used for code" AST / CFG+DFG / heterogeneous split + operator split (GGNN/GCN/GAT/MPNN)

Trajectory Lists papers chronologically code2vec (path) → Allamanis2018 (GGNN on graph) → Devign (multi-edge) → GraphCodeBERT (fusion)

Task grounding None 5-task table with graph type per task

[12] provenance "Proc. ICLR 2017" arXiv:1609.02907 + ICLR 2017 Workshop (not main track)

Open problems "More datasets needed" Scalability / cross-language edge semantics / interpretability / pretraining objective

5 fatal signals (IEEE × CS survey)

1. Mixing (Author, Year) into an IEEE survey—numeric only.

2. Alphabetizing the reference list—IEEE numbers by citation order.

3. Citing code2vec/code2seq as "GNN papers"—they are path-attention baselines, not GNNs; the GNN bridge is Allamanis 2018 [3].

4. Listing Devign as "GCN" only—it uses a typed-edge GNN, not vanilla GCN.

5. Citing GCN [12] as "in Proc. ICLR"—it is an arXiv technical report presented at an ICLR workshop, never a main-track paper.

6. et al. after 3 authors in References under IEEE-2018—must list all 6, then et al. only if 7+.

Rewrite Checklist (IEEE survey)

1. Open with the structure-aware shift (sequence → AST → graph), cite [1] as the umbrella survey.

2. Build §2 graph construction as three sub-classes (AST-path / flow-augmented / heterogeneous), not a paper list.

3. Build §3 operator layer (GGNN→GCN→GAT→MPNN) citing general GNN survey [11] and GCN origin [12].

4. Ground every graph type in a task table (§4).

5. Citations numeric [1], assigned in first-appearance order; never back-reference alphabetically.

6. References: initials-first, "Title," Journal, vol/no/pp/year, doi: or arXiv:.

7. Verify [12] provenance—arXiv:1609.02907, ICLR 2017 Workshop, not main-track.

8. Close with 4 named open problems, not "future work is needed."

9. Verify every arXiv ID and DOI resolves.

FAQ

Why numeric citations instead of (Author, Year)? IEEE CS venues (TSE, TPAMI, ICLR/NeurIPS proceedings reprints, TOSEM) use numeric [1] in citation order so the References section compresses to a numbered list. This is the single biggest format discontinuity from the Harvard/APA pages in this batch.

Is code2vec a GNN model? No. code2vec [4] and code2seq [5] aggregate AST path contexts with attention; there is no message passing between nodes. They are the immediate predecessor to code GNNs—Allamanis et al. 2018 [3] is the first to put a GGNN on a code graph with added next-token/last-use edges.

What is the difference between Devign and GraphCodeBERT? Devign [6] is a pure GNN on a heterogeneous AST+CFG+DFG+PDG graph for vulnerability detection. GraphCodeBERT [9] is a transformer pretrained with a data-flow graph as auxiliary attention guidance—not a GNN, but it borrows the data-flow graph construction that code-GNN surveys must cover. A survey that omits one of them misses half the trajectory.

Is Kipf & Welling's GCN an ICLR main-track paper? No. It was released as arXiv:1609.02907 (Univ. of Amsterdam technical report, 2017) and presented at the ICLR 2017 Workshop on Representation Learning on Graphs and Manifolds. It is not an ICLR main-track paper, so cite it as the arXiv report—not "in Proc. ICLR."

Can I reuse these 13 references? No. Use the survey structure and IEEE mechanics as a model; run your own search and confirm every arXiv ID / DOI.

Continue the research workflow with Acade

Acade can help you search for literature, organize evidence notes, build a synthesis outline, and revise citation-supported academic writing. It cannot guarantee source accuracy or replace your reading, methodological judgment, institutional rules, or responsibility for the submitted work.

Tell Acade your topic, discipline, citation style, evidence requirements, and assignment constraints, then ask it to help organize the next verifiable research step.