English for Writing Research Papers

Summary

Part I Writing Skills

1. Planing and Preparation

  • Consult with your professor and colleagues about the most appropriate journal
    where you can publish your research
  • Match your topic to the journal, or vice versa
  • Download the guidelines for authors – these will tell you about the style and
    structure of your paper
  • Choose frequently cited papers in the journal to see how other authors construct
    their argumentation, and note down ways in which your research is different and
    innovative with respect to theirs
  • Choose one paper as a model onto which to map your research, imitating the style
    and organization. This model should be written by a native English speaker
  • Note down useful / standard phrases from your model paper which you can then
    use in your own paper
  • Decide on the best order to write the various sections of your paper.
    It is generally best to start with a very rough draft of the Abstract, and then whichever
    section is clearest in your head (generally the Materials and Methods)
  • Consider having separate documents for each section. This enables you to work
    on several sections at the same time
  • Make sure your unique contribution to your community is very clear in every
    section, not just in the Abstract
  • Write in a way that even a non-expert can understand
  • Referees work for free and often outside working hours – never submit a carelessly written manuscript
  • Access referees report forms to understand the ways that referees will evaluate
    your work
  • Write directly in English, and use every opportunity for improving your writing
    skills
  • Use online resources

2. Word Order

  • Basic English word order is: (1) subject, (2) verb, (3) direct object, (4) indirect
    object. Keep these four elements in this order and as close to each other as
    possible
  • If you have a choice of subjects, choose the one that is the most relevant and
    leads to the shortest construction
  • Avoid delaying the subject. So don’t begin a sentence with the impersonal it
  • Avoid inserting parenthetical information between the subject and the verb
  • Most adverbs are located just before the main verb, and before the second auxiliary verb when there are two auxiliaries
  • If possible, delay adverbs until later in the sentence. The main exceptions to this
    rule are adverbs of contrast and those that enumerate points
  • Put adjectives before the noun they describe, or use a relative clause. Do not
    insert an adjective between two nouns or before the wrong noun
  • Do not indiscriminately put nouns in a string
  • Avoid ambiguous word order

3. Breaking Up Long Sentences

  • Don’t separate the subject from its verb using more than 8–10 words
  • Avoid adding extra information to the end of the main clause, if the main clause
    is already about 15–20 words long
  • Check to make sure that a sentence has a maximum of 30 words, and don’t use
    more than three or four 30-word sentences in the whole paper
  • Consider beginning a new sentence if the original sentence is long and contains
    one or more of the following (or equivalents): and, which, a link word, the -ing
    form, in order to
  • Maximize the use of periods (.). Use the minimum number of commas (,), avoid
    semicolons (;) and parentheses
  • Don’t worry about repeating key words. If dividing up a long sentence into
    shorter sentences means that you have to repeat key words, this is not a problem.
    In fact this repetition will increase the clarity of your writing

4. Sturcturing Paragraphs and Sentences

  • Always think about your readers - order the information you give them in the most logical way
    and in the simplest form
  • Begin each paragraph with a topic sentence, then use the rest of the paragraph to develop this topic. If
    appropriate have a short concluding sentence at the end of the paragraph
  • Decide whether to begin a new section with a short summary, or whether to go directly to the main points.
  • Put the topic as the subject of the paragraph or sentence, then give known information(context,background)
    fllowed by new information. Consider not giving the known information if it will be obvious for you readers.
  • Move from the general to the incresingly specific, do not mix the two.
  • Always progress in the most logical and consistent oreder, do not go backwards and forwards
  • Don’t force readers to change their perspective: put negations and qualifying phrase at ore near the beginning of
    a sentence
  • Break up long paragrahps and begin a new paragraph when you talk about your study and your key findings
  • Avoid redundancy in the final paragraph of a section

5. Being Concises and Removing Redundancy

  • Deleting any words that are not 100% necessary
  • Finding ways of expressing the same concept with fewer words
  • Using verbs rather than nouns
  • Choosing the shortest words and expressions
  • Avoiding impersonal phrase that begin it is …

6. Avoiding Ambiguity and Vagueness

  • which is used for adding information about the preceding noun, that defines the preceding noun
  • which, that and who should noly refer to the noun immediately preceding them
  • the -ing form (gerund) has no subject. Make sure it is clear what the subject of the -ing form is.
  • clarify whether something is a consequence of doing something or a means to do something by using thus (consequence)
    and by (means) before the -ing form
  • use the definite article(the)before a noun only if you refer to a specific example of that noun. If you are giving a generic
    idea, do not use the article
  • learn the most frequent uncountable nouns and false friends in your field
  • be very careful when you use pronouns (this,that,them,it etc,) - make sure it is clear what they refer to and don’t be afraid of repeating the same word many times(if this will improve clarity)
  • avoid using the former … the latter, simply repeat the related noun
  • if necessary specify exact locations, when using above and below
  • use respectively when it is ont 100% clear how items are related to each other
  • be careful of punctuation with which and and - punctuation must help the reader understand the relationships between the various parts of the sentence
  • don’t confuse both … and (inclusive) with either… or (exclusive); and i.e (definitons) and e.g (examples)
  • never use synonyms for key words, only for generic verbs and adjectives
  • use the most precise word possible

7. Clarifying Who Did What

  • Follow the journal’s instructions regarding whether you can use we/ I or if you have to use the passive at all times
  • You may have the impression that the passive form is considered to be more elegant in scientific papers. Whether this impression is true or not, be aware that the passive inevitably creates problems for your readers because it may be diffcult for them to know immediately and with certainty whether you or another author made a particular finding.
  • Do not rely on a reference to a figure or a table, or a reference to the bibiography to distinguish your new data from those in the literature. Make sure the reference clearly indicates it is another author’s work and not a previous paper by you.
  • Be aware that if you make mistakes in the usage of tense when you are comparing your work with other authors’ work, you could really confuse your readers. Make sure you consistently use the correct tense and remember that in English there is a real difference betweeen the SIMPLE PAST (finished actions with time indication) and the PRESWNR PERFECT (pase to present actions, finished actions with no time indication)
  • Avoid using we when it is not really necessary, i.e. to explain your train of thought.
  • Help readers to distinguish between your work and others by using a series of short paragraphs, rather than one long paragraph.
  • If you mention another author’s paper, make sure that the reader understands why you are mentioning that paper and how it relates to your own work.

8. Highlighting Your Findings

  • Be aware of how the layout of your paper can affect where readers focus their eyes - break up long blocks of text using shorter paragraphs and figures/tables
  • Begin a new paragraph when highlighting something important
  • Use shorter sentences and paragraphs to make your key points
  • Use more dynamic language - make sure the reader understands immediately that you are about to say something important
  • Don’t just tell the readers that something is important - show them
  • Tell your readers the implications of your findings
  • Talk about your weakness not just your strengths; do not make the referees suspect any bias in your work

    9. Hedging and Criticising

  • Tone down verbs, adjectives, adverbs and your general level of certainty.
  • Be aware that the ways you express uncertainty may simply not translate into English.
  • Provide alternative interpretations of your data.
  • Tell the reader from which standpoint you want them to interpret or judege your data.
  • Use impersonal forms to distance yourself when interpreting your findings.
  • Save your face by writing in an impersonal fashion.
  • Try to put the work of authors in a positive light. If appropriate say their work is open to another interpretation(i.e. yours).
  • Dont’t overhedge.
  • Consider getting help from a native speaker when hedging your claims.

10.Plagiarising and Plagiarism

  • Plagiarism is a serious issue in international science, even though it may not be considered so in your country of origin. It is easy for native speakers to spot it in the work of non native speakers. If you commit plagiarism your credibility and reputation will be seriously compromised. If you not sure whether you have plagiarized your own or someone else’s work, use CrossCheck
  • Copying phrases from other people’s work is perfectly acceptable and is a good way to learn useful phrases in English that you can then use in your own work. However. such phrases must be 100% generic in the sense that they hold absolutely no hard information
  • Use direct quotations sparingly. The problem is that the referee (or your professor) cannot be sure that you have fully understood the quotation
  • Typical ways to paraphrase:
    1. use of synonsyms for non key words (especially verbs, adverbs and adjectives)
    2. change of part of speech, for example: from noun to verb, from noun to adjective, from one category of noun to another category of noun (e.g. science to scientist)
    3. change of nouns and pronouns from singular to plural and vice versa
    4. change of verb form, for example: from -ing form to infinitive, from simple to continuous, from active to passive
    5. change of style from personal to impersonal
    6. reversal of the order in which information is presented
  • Never paraphrase technical words
  • If the orginal contains ideas that in some sense ‘belonged’ to the original author, then this author should be acknowledged. This is true even if you have radically changed the orginal so that it is now unrecognizable
  • When quoting the work of a ‘third’ author, cite the reference to that third author’s paper

Part II Sections of a Paper

11. Titles

  • You need to check that your title is:
    1. in correct English - in terms of syntax, vocabulary, spelling and capitalization
    2. understandable(no strings of nouns)
    3. eye-catching and dynamic(through effective use of vocabulary and even punctuation)
    4. sufficiently and appropriately specific
    5. reflects the content of your paper
    6. expressed in a form that is acceptable for a journal
  • You can check the syntax and the level of understandability by consulting with a native speaker. Generally speaking titles that contain at least one verb and one or more prepositions tend to be much easier to understand.
  • You can check the syntax and the level of understandability by consulting with a native speaker. Generally speaking titles that contain at least one verb and one or more prepositions tend to be much easier to understand.
  • You can check the vocabulary and spelling using Google Scholar. Remenber that an automatic spell check is not enough.
  • The best way to decide whether it is eye-catching and sufficiently specific is to prepare several titles(including ones in two parts, and in the form of a question) with various levels of specificity and ask colleagues to choose their favorite.
  • Unless you get someone to read the whole paper for you, you are probably the best judge of whether your title reflects the actual content of your paper. If it doesn’t, the referees will probably tell you.

12. Abstract

  • Have I followed the journal’s instructions to authors? Have I followed the right structure(i.e. structured, unstructured) and style(we vs passive)?
  • Have I convered the relevant points from those below?
    1. background / context
    2. research problem/aim - the gap I plan to fill
    3. methods
    4. results
    5. implications and/or conclusions
  • Have I chosen my keywords carefully so that readers can locate my Abstrct?
  • Whenever I have given my readers information, will it be 100% clear to them why they are being given this information?(You know why, but they don’t.)
  • Can I make my Abstract less redundant? If I tried to reduce it by 25% would I really lose any key content?
  • Have I used tense correctly? PRESENT SIMPLE(established knowledge), PRESENT PERFECT(past to present background information), PAST SIMPLE(my contribution)

13. Introduction

  • To make a self-assessment of your Introduction, you can ask yourself the following questions.
    1. Is my research question clear?
    2. Does my Introduction act as a clear road map for understanding my paper?
    3. Is it sufficiently different from the Abstract, without any cut and pastes(some overlap is fine)
    4. Have I mentioned only what my readers specifically need to know and what I will subsequently refer to in the Discussion?
    5. Have I been as concise as possible?
    6. Have I used tenses correctly? PRESENT SIMPLE(general background context, description of what will be done in the paper), PRESENT PERFECT (past to present solutions), PAST SIMPLE(my contribution, though this may also be expressed using the PRESENT SIMPLE or FUTURE SIMPLE)

14. Literature Review

  • To make a self-assessment of your Literature Review, you can ask yourself the following questions
    1. Have I mentioned only what my readers specifically need to know and what I will subsequently refer to in the Discussion?
    2. Are the papers I have mentioned in a logical order? Is it clear why i have chosen these papers and not others?
    3. Have I followed my journal’s instructions regarding how I make references to the literature? Where possible have I done this in a variety of ways?
    4. Have I removed any redundancy when reporting the literature?
    5. Have I used the tense correctly? PRESENT PERFECT(at the beginning of review to give general overview; for past-to-present evolutions), PAST SIMPLE(when specific dates are mentioned within a sentence; for the verbs that introduce an author’s findings)

15. Methods

  • To make a self-assessment of your Methods section, you can ask yourself the following questions

    1. Have I really described my Methods in a way that is easy for readers to follow and which would enable them to replicate my work? Have I ensured that I have covered every step? Is my structure clear and complete?
    2. Have I been as concise as possible? Have I used references to previous works rather than repeating descriptions that readers could easily find elsewhere?
    3. Do the individual sentences in each paragraph contain too many, too few, or just the right manageable number of steps? Have I ensured that my sentences don’t sound like lists?
    4. Have I thought about the way readers prefer to receive information?(no ambiguity, no back referencing, everything in chronological order, headings, bullets)?
    5. Have I checked my grammar(infinitive, gerund, allow, thus etc.) with regard to how I outline how and why I made certain choices?
    6. Have I used tenses correctly? PAST SIMPLE(in the passive form to describe what I did), PRESENT SIMPLE(descriptions of established scientific fact)

    16. Results

    • To make a self-assessment of your Results section, you can ask yourself the following questions.

      1. Have I expressed myself as clearly as possible, so that the contribution that my results give stands out for the referees and readers?
      2. Have I limited myself to only reporting the key result or trends that each figure and table conveys, rather than reiterating each value?
      3. Have I avoided drawing conclusions?(this is only true when the Results is an independent section)
      4. Have I chosen the best format to present my data(e.g. figure or table)? Have I ensured that this is no redundancy between the various figures and tables?
      5. Have I ensured that my tables of results are comprehensive in the sense that they do not exclusively include points that prove my point?
      6. Have I mentioned only what my readers specifically need to know and what I will subsequently refer to in the Discussion?
      7. Have I mentioned any parts of my methodology (e.g. selection and sampling procedures) that could have affected my results?
      8. Have I used tenses correctly? PAST SIMPLE for your findings(in the passive form), PRESENT SIMPLE(descriptions of established scientific fact)

      17.Discussion

      • When you have finished writing your Discussion, it is a good idea to make sure you can honestly answer ‘yes’ to all the questions below. This will enable your peers to make a critical assessment with regard to the strengths and weaknesses of (a) how you carried out your research (b) and how your analyzed your findings. The result will be that you will be seen as a credible researcher.
      1. Is my contribution to the knowledge gap clear? Have I underlined the significance of my findings?
      2. Have I explained what I believe to be new and important very clearly but without exaggerating? Have I ensured that I have not over-interpreted my results(i.e. attributed interpretations to them that cannot actually be supported)?
      3. Have I truly interpreted my results, rather than just reiterating them? Have I shown the relationship(confirmation or rejection) between my results and my original hypothesis? Have I generated new theory rather than simply giving descriptions?
      4. Is there a good balance, rather than being a one-sided version? Have I really offered alternative explanations?
      5. Have I clearly distinguished fact from speculation? Will the reader easily be able to understand when I am merely suggesting a possible interpretation rather than providing conclusibe evidence for something?
      6. Have I ensured that there is no bias in my research?(i.e. I have not hidden any of my data or any unexpected results, simply because they do not confirm what I was hoping to find)
      7. Have I included those works in the literature that do not corroborate my findings Likewise, have I avoided distorting the magnitude or direction of the data of the literature that I have selected?(i.e. I have made sure that I have not committed publication bias)
      8. Have I discussed my findings in the context of what I said in the Introduction? Have I exploited my Review of the Literature?
      9. Have I integrated my results with previous research(including my own) in order to explain what I obesrved or found?
      10. Have I intergrated my results with previous research (including my own) in oreder to explain what I observed or found?
      11. Have my criticisms of the literature been justified and constructive?
      12. Have I ensured that I have not introduced any new findings(i.e. findings not mentioned in the Results)?
      13. Are all the statements I have made in the text supported by the data contained in my figures and tables?
      14. Have I removed any trivial information? Have I been as concise as possible?

      In addition, remember to make a clear distinction between your work and others but appropriate use of
      we/our, they/their
      references in parentheses to the literature
      minimal use of passive form

      You can massively improve the structure and the language you use in your Discussion by analyzing how other authors in your field write their Discussion sections. If possible, try to adopt the same approach to analyzing texts as I have used in this chapter.

      18. Conclusions

      • To make a self-assessment of your Conclusions, you can ask yourself the following questions.
      1. Is what I have written really a Conclusions section?(If it is more than 200-250 words, then it probably isn’t - it needs to be much shorter)
      2. If the conclusions are included in the Discussion, have I clearly signaled to the reader that I am about to discuss my conclusions(e.g. by writing In conclusion …)?
      3. Have I given a maximum of one line to comments related to descriptions of procedures, methodology, interviews etc.?(Generally such comments are not needed at all, unless the primary topic of your paper is the methodology itself)
      4. Have I avoided cut and pastes from earlier sections? Do my Conclusions differ appropriately from my Abstract, Introduction and final paragraph of my Discussion?
      5. Is my work as complete as I say it is?(i.e. I am not trying to get priority over other authors by claiming inferences that cannot really be drawn at this stage)
      6. Have I introduced new avenues of potential study or explained the potential impact of my conclusions? Have I ensured that I have only briefly described these future avenues rather than getting lost in detail?
      7. Are the possible applications I have suggested really feasible? Are my recommendations appropriate?
      8. Have I used tenses correctly? PRESENT PERFECT(to describe what you have done during the writing process), PAST SIMPLE(what you did in the lab, in the field, in your survey etc.)

      In addition, you should look at the summary questions for the Discussion, as these may also be helpful in deciding whether your Conclusions will have the necesary impact on your readers

Probabilistic Graphical Models

1. Foundations

1.1 Probability Theory

1.1.1 Probability Distirbutions

1.1.2 Basic Concepts in Probability

1.1.3 Random Variables and Joint Distributions

1.1.4 Independence and Conditional Independence

1.1.5 Querying a Distribution

1.1.6 Continuous Space

1.1.7 Expectation and Variance

1.2 Graphs

1.2.1 Nodes and Edges

1.2.2 Subgraphs

1.2.3 Paths and Trails

1.2.4 Cycles and Loops

Representation

2. The Bayesian Network Representation

2.1 Exploting Independence Properties

2.1.1 Independent Random Variables

2.1.2 The Conditional Parameterization

2.1.3 The Naive Bayes Model

2.2 Bayesian Network

2.2.1 The Student Example Revistited

2.2.2 Basic Independencies in Bayesian Networks

2.2.3 Graphs and Distributions

2.3 Independencies in Graphs

2.3.1 D-separation

2.3.2 Soundness and Completeness

2.3.3 An Algorithm for d-Separation

2.3.4 I-Equivalence

2.4 From Distributions to Graphs

2.4.1 Minimal I-Maps

2.4.2 Perfect Maps

2.4.3 Finding Perfect Maps (#)

3. Undirected Graphical Models

3.1 The Misconception Example

3.2 Parameterization

3.2.1 Factors

3.2.2 Gibbs Distributions and Markov Networks

3.2.3 Reduced Markov Networks

3.3 Markov Netwrok Independencies

3.3.1 Basic Independencies

3.3.2 Independencies Revisited

3.3.3 From Distributions to Graphs

3.4 Parameterization Revisited

3.4.1 Finer-Grained Parameterization

3.4.2 Overparameterization

3.5 Bayesian Networks and Markov Networks

3.5.1 From Bayesian Networks to Markov Networks

3.5.2 From Markov Networks to Bayesian Networks

3.5.3 Chordal Graphs

3.6 Partially Directed Models

3.6.1 Conditional Random Fields

3.6.2 Chain Graph Models (#)

4. Local Probabilistic Models

4.1 Tabular CPDs

4.2 Deterministic CPDs

4.3 Context-Specific CPDs

4.4 Independence of Causal Influence

4.4.1 The Noisy-Or Model

4.4.2 Generalized Linear Models

4.4.3 The General Formulation

4.5 Continuous Variables

4.5.1 Hybrid Models

4.6 Conditional Bayesian Networks

5. Template-Based Representations

5.1 Temporal Models

5.1.1 Basic Assumptions

5.1.2 Dynamic Bayesian Networks

5.1.3 State-Observation Models

5.2 Template Variables and Template Factors

5.3 Directed Probabilistic Models for Object-Relational Domains

5.3.1 Plate Models

5.3.2 Probabilistic Realtional Models

5.4 Undirected Representation

5.5 Structural Uncertainty (#)

5.5.1 Relational Uncertainty

5.5.2 Object Uncertainty

6. Gaussian Network Models

6.1 Multivariate Gaussians

6.1.1 Basic Parameterization

6.1.2 Operations on Gaussians

6.1.3 Independencies in Gaussians

6.2 Gaussian Baysian Networks

6.3 Gaussian Markov Random Fiels

7. The Exponential Family

7.1 Exponential Families

7.1.1 Linear Exponential Families

7.2 Factored Exponential Families

7.2.1 Product Distributions

7.2.2 Bayesian Networks

7.3 Entropy and Relative Entropy

7.3.1 Entropy

7.3.2 Relative Entropy

Infernece

8. Exact Inference: Variable Elimination

8.1 Analysis of Complexity

8.1.1 Analysis of Exact Inference

8.1.2 Analysis of Approximate Infernece

8.2 Variable Elimination: The Basic Ideas

8.3 Variable Elimination

8.3.1 Basic Elimination

8.3.2 Dealing with Evidence

8.4 Complexity and Graph Structure: Variable Elimination

8.4.1 Simple Analysis

8.4.2 Graph-Theoretic Analysis

8.4.3 Finding Elimination Orderings (#)

8.5 Conditioning (#)

8.5.1 The Conditioning Algorithm

8.5.2 Conditioning and Varialble Elimination

8.5.3 Graph-Theoretic Analysis

8.5.4 ImProved Conditioning

8.5 Inference with Structured CPDs (#)

8.5.1 Independence of Causal Influence

8.5.2 Context-Specific Independence

9. Exact Inference: Clique Trees

9.1 Variable Elimination and Clique Trees

9.1.1 Cluster Graphs

9.1.2 Clique Trees

9.2 Message Passing: Sum Product

9.2.1 Variable Elimination in a Clique Tree

9.2.2 Clique Tree Calibration

9.2.3 A Calibrated Clique Tree as a Distribution

9.3 Message Passing: Belief Update

9.3.1 Message Passing with Division

9.3.2 Equivalence of Sum-Product and Belief Update Messages

9.3.3 Answering Queries

9.4 Constructiong a Clique Tree

9.4.1 Clique Trees from Variable Elimination

9.4.2 Clique Trees form Chordal Graphs

10. Inference as Optimization

10.1 Introduction

10.1.1 Exact Inference Revisited (#)

10.1.2 The Energy Functional

10.1.3 Optimizing the Energy Functional

10.2 Exact Inference as Optimization

10.2.1 Fixed-Poing Characterization

10.2.2 Inference as Optimization

10.3 Propagation-Based Approximation

10.3.1 A Simple Example

10.3.2 Cluster-Graph Belief Progagation

10.3.3 Properties of Cluster-Graph Belief Propagation

10.3.4 Analyzing Convergence (#)

10.3.5 Constructiong Cluster Graphs

10.3.6 Variatioanal Analysis

10.3.7 Other Enrtopy Approximations (#)

10.4 Propagation with Approximate Messages (#)

10.4.1 Facotrized Messages

10.4.2 Approximate Message Computation

10.4.3 Inference with Approximate Messages

10.4.4 Expectation Propagation

10.4.5 Variational Analysis

10.5 Structured Variatioanl Approximations

10.5.1 The Mean Field Approximation

10.5.2 Structured Approximations

10.5.3 Local Variational Methods (#)

11. Particle-Based Approximate Inference

11.1 Forward Sampling

11.1.1 Sampling from a Bayesian Network

11.1.2 Analysis of Error

11.1.3 Conditional Probability Queries

11.2 Likelihood Weighting and Importance Sampling

11.2.1 Likelihood Weighting: Intuition

11.2.2 Importance Sampling

11.2.3 Importance Sampling for Bayesian Networks

11.2.4 Importance Sampling Revisisted

11.3 Markov Chain Monte Carlo Mehtods

11.3.1 Gibbs Sampling Algorithm

11.3.2 Markov Chains

11.3.3 Gibbs Sampling Revisited

11.3.4 A Broader Class of Markov Chains (#)

11.3.4 Using a Markov Chain

11.4 Collapsed Particles

11.4.1 Collapsed Likelihood Weighting (#)

11.4.2 Collapsed MCMC

11.5 Deterministic Search Methods (#)

12. MAP Inference

12.1 Overview

12.1.1 Computational Complexity

12.1.2 Overview of Solution Methods

12.2 Variable Elimination for (Marginal) MAP

12.2.1 Max-Product Variable Elimination

12.2.2 Finding the Most Probalbe Assignment

12.2.3 Variable Elimination for Marginal MAP (#)

12.3 Max-Product in Clique Trees

12.3.1 Computing Max-Marginals

12.3.2 Message Passing as Reparameterization

12.3.3 Decoding Max-Marginals

12.4 Max-Product Belief Propagation in Loopy Cluster Graphs

12.4.1 Standard Max-Product Message Passing

12.4.2 Max-Product BP with Counting Numbers (#)

12.5 MAP as a Linear Optimization Problem

12.5.1 The Integer Program Formulation

12.5.2 Linear Programming Relaxation

12.5.3 Low-Temperature Limits

12.6 Using Graph Cuts for MAP

12.6.1 Inference Using Graph Cuts

12.6.2 Nonbinary Variables

12.7 Local Search Algorithms

13. Inference in Hybrid Networks

13.1 Variable Elimination in Gaussian Networks

13.1.1 Canonical Forms

13.1.2 Sum-Product Algorithms

13.1.3 Gaussian Belief Propagation

13.2 Hybird Networks

13.2.1 The Difficulities

13.2.2 Factor Operations for Hybrid Gaussian Networks

13.2.3 EP for CLG Networks

13.2.4 An “Exact” ClG Algorithm (#)

13.3 Nonlinear Dependencies

13.3.1 Linearization

13.3.2 Expectation Propagation with Gaussian Approximation

13.4 Particle-Based Approximation Methods

13.4.1 Sampling in Continuous Spaces

13.4.2 Forwrad Sampling in Bayesian Networks

13.4.3 MCMC Methods

13.4.4 Collapsed Particles

13.4.5 Nonparametric Message Passing

14. Inference in Temporal Models

14.1 Exact Inference

14.1.1 Filtering in State-Observation Models

14.1.2 Filtering as Clique Tree Propagation

14.1.3 Clique Tree Inference in DBNs

14.1.4 Entanglement

14.2 Approximate Inference

14.2.1 Key Ideas

14.2.2 Factored Belief State Methods

14.2.3 Particle Filtering

14.2.4 Deterministic Search Techniques

14.3 Hybrid DBNs

14.3.1 Continuous Models

14.3.2 Hybrid Models

15. Learning Graphical Models: Overview

15.1 Goal of Learning

15.1.1 Density Estimation

15.1.2 Specific Prediction Tasks

15.1.3 Knowledge Discovery

15.2 Learning as Optimization

15.2.1 Empirical Risk as Overfitting

15.2.2 Discriminative versus Generative Training

15.3 Learning Tasks

15.3.1 Model Constraints

15.3.2 Data Observability

15.3.3 Taxonmomy of Learning Tasks

16. Parameter Estimation

16.1 Maximum Likelihood Estimation

16.1.1 The Thumbtack Example

16.1.2 The Maximum Likelihood Principle

16.2 MLE for Bayesian Networks

16.2.1 A Simple Example

16.2.2 Global Likelihood Decomposition

16.2.3 Table-CPDs

16.2.4 Gaussian Bayesian Networks (#)

16.2.5 Maximum Likelihood Estimation as M-Projection

16.3 Bayesian Parameter Estimation

16.3.1 Parameter Independence and Global Decompotion

16.3.2 Local Decompostion

16.3.3 Priors for Baysian Network Learning

16.3.4 MAP Estimation (#)

16.4 Learning Models with Shared Parameters

16.4.1 Global Parameter Sharing

16.4.2 Local Parameter Sharing

16.4.3 Bayesian Inference with Shared Parameters

16.4.4 Hierarchical Priors (#)

16.5 Generalization Analysis

16.5.1 Asymptotic Analysis

16.5.2 PAC-Bounds

17. Structure Learning in Baysian Networks

17.1 Constraint-Based Approcaches

17.1.1 General Framework

17.1.2 Independence Tests

17.2 Structure Scores

17.2.1 Likelihood Scores

17.2.2 Bayesian Score

17.2.3 Marginal Likelihood for a Single Variable

17.2.4 Bayesian Score for Bayesian Networks

17.2.4 Understanding the Bayesian Score

17.2.5 Priors

17.2.6 Score Equivalence (#)

17.3.1 Learning Tree-Structured Networks

17.3.2 Known Order

17.3.3 General Graphs

17.3.4 Learning with Equivalence Classes (#)

17.4 Bayesian Model Averaging (#)

17.4.1 Basic Theory

17.4.2 Model AVeraging Given an Order

17.4.3 The General Case

17.5 Learning Models with Additional Structure

17.5.1 Learning with Local Structure

17.5.2 Learning Template Models

18. Partially Observed Data

18.1 Foundations

18.1.1 Likelihood of Data and Observation Models

18.1.2 Decoupling of Observation Mechanism

18.1.3 The Likelihood Function

18.1.4 Identifiability

18.2 Parameter Estimation

18.2.1 Gradient Ascent

18.2.2 Expectation Maximization (EM)

18.2.3 Comparison: Gradient Ascent versus EM

18.2.4 Approximate Inference (#)

18.3 Bayesian Learning with Incomplete Data (#)

18.3.1 Overview

18.3.2 MCMC Sampling

18.3.3 Variational Bayesian Learning

18.4 Structure Learning

18.4.1 Scoring Structures

18.4.3 Structural EM

18.5 Learning Models with Hideen Variables

18.5.1 Information Content of Hidden Varialbles

18.5.2 Determining the Cardinality

18.5.3 Introducing Hidden Variables

19. Learning Undirected Models

19.1 Overview

19.2 The Likelihood Function

19.2.1 An Example

19.2.2 Form of the Likelihood Function

19.2.3 Properties of the Likelihood Function

19.3 Maximum (Conditiaonal) Likelihood Parameter Estimation

19.3.1 Maximum Likelihood Estimation

19.3.2 Conditionally Trained Models

19.3.3 Learning with Missing Data

19.3.4 Maximum Entropy and Maximum Likelihood (#)

19.4 Parameter Priors and Regularization

19.4.1 Local Priors

19.4.2 Global Priors

19.5 Learning with Approximate Inference

19.5.1 Belief Propagation

19.5.2 Map-Based Learning (#)

19.6 Alternative Objectives

19.6.1 Pseudolikelihood and Its Generalizations

19.6.2 Constrastive Optimization Criteria

19.7 Structure Learning

19.7.1 Structure Learning Using Independence Tests

19.7.2 Score-Based Learning: Hypothesis Spaces

19.7.3 Objective Functions

19.7.4 Optimization Task

19.7.5 Evaluating Changes to the Model

20. Causality

20.1 Motivation and Overview

20.1.1 Conditioning and Intervention

20.1.2 Correlation and Causation

20.2 Causal Models

20.3 Structural Causal Identifiability

20.3.1 Query Simplification Rules

20.3.2 Interated Query Simplification

20.4 Mechanisms and Response Variables (#)

20.5 Partial Identifiability in Functional Causal Models (#)

20.6 Counterfactual Queries (#)

20.6.1 Twinned Netwroks

20.6.2 Bounds on Counterfactual Queries

20.7 Learning Causal Models

20.7.1 Learning Causal Models without Confounding Factors

20.7.2 Learning from Interventional Data

20.7.3 Dealing with Latent Variables (#)

20.7.4 Learning Functional Causal Models (#)

21. Utilities and Decisions

21.1 Foundations: Maximizing Expected Utility

21.1.1 Decision Making Under Uncertainty

21.1.2 Theoretical Justification (#)

21.2 Utility Curves

21.2.1 Utility of Money

21.2.2 Attitudes Toward Risk

21.2.3 Rationality

21.3 Utility Elicitation

21.3.1 Utility Elicitation Procedures

21.3.2 Utility of Human Life

21.4 Utilities of Complex Outcomes

21.4.1 Preference and Utility Independence (#)

21.4.2 Additive Independence Properties

22. Structured Decision Problems

22.1 Decision Trees

22.1.1 Representation

22.1.2 Backward Induction Algorithm

22.2 Influence Diagrams

22.2.1 Basic Representation

22.2.2 Decision Rules

22.2.3 Semantics and Optimality Criterion

22.3 Backward Induction in Influence Diagrams

22.3.1 Decision Trees for Influence Diagrams

22.3.2 Sum-Max-Sum Rule

22.4 Computing Expected Utilities

22.4.1 Simple Variable Elimination

22.4.2 Multiple Utility Variables: Simple Approaches

22.4.3 Generalized Variable Elimination (#)

22.5 Optimization in Influence Diagrams

22.5.1 Optimizaing a Single Decision Rule

22.5.2 Iterated Optimization Algorithm

22.5.3 Strategic Relevance and Global Optimality

22.6 Ignoring Irrelevant Information

22.7 Value of Information

22.7.1 Single Observations

22.7.2 Multiple Observations

Welcome to Hexo! This is your very first post. Check documentation for more info. If you get any problems when using Hexo, you can find the answer in troubleshooting or you can ask me on GitHub.

Quick Start

Create a new post

1
$ hexo new "My New Post"

More info: Writing

Run server

1
$ hexo server

More info: Server

Generate static files

1
$ hexo generate

More info: Generating

Deploy to remote sites

1
$ hexo deploy

More info: Deployment