Dr. J's Graduate Seminar Lecture Notes

(C) Copyright 2021-23 by Clinton Jeffery, for use in Dr. J's Graduate Seminar classes only.

lecture #1 began here

Syllabus

Let's go over the syllabus.

Announcements

Homework #1: Responsible Conduct of Research

Reading

What is Research? Why do we do it?

Everyone please go around the room and tell me:







Student definitions of research included

Academic Research

Just as a reminder, when we talk about research this semester, we are talking about

Types of computer science research

theory
The royal house of CS. We are a branch of (applied) mathematics. What about theoretical CS research is distinct from math?
systems
CS is a branch of engineering. Is CS systems research the same as other forms of engineering research? Maybe it is one meta-level more wide open.
applications?
is anything that uses advanced computing computer science research?
these categories ...not really mutually exclusive
the best theoreticians I have ever met, tested theories by writing real world implementations.
Now,

lecture #2 began here

Schedule Not Moving to Wednesday Evenings

Very sorry, but we will not be moving the Wednesday class to 5:30pm, as it would introduce a conflict for a student in the class.

First Homework (RCR) Update

Someone had some difficulty getting onto the CITI training, so I looked into it a bit further, with the following result:

Speakers

CS Research Training

Choosing an Area within Which to Seek a Topic

When I started writing this section, I was thinking about how I first became interested in my chosen research field. Then I realized that professionals working in the field face similar choices.
natural inclination
"I just like it"
(pseudo)heredity
"my aunty studied it"
bribery
A Fountain of Money will pay me to research it
path of least resistance
I went into this topic to avoid pain
compassion
I went into it to reduce others' pain
inertia
I'll keep researching in the area I'm already familiar with
bandwagoning
I heard an inspiring talk
herd instinct
A lot of people said this topic was trending
random
I picked this area by throwing darts one night in a bar

Your Research Area (at least for today)

If you don't have a research area yet, I can pick one for you! Uh oh... Don't worry, this is for HW#1, you can change your area in future homeworks

"How to Read a Paper" paper

My comments:

Literature Survey (from [Keshav])

  1. Use Search Engine to find 3-5 recent papers in chosen area
  2. Do pass 1 on each, then read related work sections
  3. If you find a recent survey paper, you are done!
  4. Find shared citations and repeated authors in your 3-5 papers' references.
  5. Add the papers who are cited in more than one paper to your pool.
  6. Go to the repeated authors' web pages (uni. home pages, google scholar pages, researchgate, ...) and look for any more recent related work.
  7. identify top conferences in the field from repeated authors' publication lists.
  8. go to top conferences in the field and read through the table of contents for their recent (say, 5 or 10 years) proceedings. Look for papers related to your chosen area. Add those papers to your pool.

HW#1

lecture #3 began here

Authorship (RCR thoughts)

Authorship Anecdotes

Things I have experienced or witnessed first- or second-hand

Looking ahead through our textbook

What the book is missing, of course is the hard part:
how to actually DO the research to obtain the answer to a difficult problem.

I have on several occasions worked with students who wanted to write the prestigious publication without actually doing the research first. Sometimes they seem unaware that a step is being skipped.
At best, they figure it out and this becomes mastery of the MPU and leads to modest career success. At worst, this is someone who really should not graduate.
I have also seen on many occasions an enormous amount of research done without successful publications. This is almost the default. Why?
Research without publication happens because...

lecture #4 began here

lecture #5 began here

Impressions from Monday's Talk

What were the strengths and weaknesses of the talk?
What was the research topic?
What were the research questions/results/contributions/evaluation?
I will ask these questions of you for each external speaker. Everyone try to contribute to each speaker discussion, so I don't have to resort to written assignments that ask you to assess and learn from the speakers.

585 is once, but colloquiums are forever

As long as you are a CS researcher, you should seek out interesting talks and learn from others. If you are here next Fall, you should come to the external talks, even though you don't have to register for 585!

Papers to Help you write a Literature Review

Homework Q&A

How can I go about accessing a paper without paying for it? I've been refining my research topic and came across the paper [XXX, behind an IEEE paywall]
  1. In many cases if you type the full title into a google search you will find that the authors have posted it publicly on their website or on a research site such as researchgate. Many organizations and copyright licenses for published works now allow this, and (for example) the federal government has long required it for research publications of their employees.
  2. If it is a major science publisher, the University might have a site license, and the paper lookup may well work from the campus network or the campus VPN. If you are a serious researcher doing a lot of these, you might get your own subscription, e.g. to the ACM Digital Library or its equivalent in other organizations whose papers you use.
  3. If you don't find it on the authors' home pages or via a google search, you might end up requesting the assistance of the university library, but that will take time.
  4. The bold sometimes have luck by literally e-mailing* the authors and asking nicely, but I don't always say yes when asked. The materials are copyrighted and it might depend on the copyright details. Your mileage with different authors will vary. *or postcards?

Chapter 2 of [Booth]

For each chapter, I'm going to ask questions like: what was the main point, what did [Booth et al] say that you hadn't already heard before, and what did you especially agree or disagree with?

Your comments:

Booth, Chapter 2

My comments:
"connecting with your reader is thinking about your reader from the beginning"
"when you report your research, you add to a 5,000 year old conversation"
"writing is an imagined conversation"
writers judge readers before they even write
whether they are aware of it or not
think about your own and your readers' roles before you begin
let's overdo it and give our readers an imaginary persona...
give your reader a reason to want to know your research results

Author Roles

This is related to the "voice" or the point-of-view that the reader perceives the author to be taking. Sleazy salesman? Stern parent?
I've found new and interesting information
I've found a solution to an important practical problem
I've found an answer to an important question

Reader Roles

Different readers come to your research with different needs/desires.
Entertain me
Help me solve my practical problem
targeted and demand-driven
Help me understand something better <-- usual academic role
objective, logical, unbiased

Checklist for Understanding your Readers

Who will read my paper?
What do they expect me to do?
How much do they know already?
How will readers respond to the answer in my main claims?

LaTeX

OK, please tell me everything you know about LaTeX:

LaTeX is a typesetting language invented by one of the luminary founders of computer science, primarily so that he could typeset his own mathematically dense, equation-laden books and papers. Some CS research outfits require LaTeX and some do not. All CS grad students should probably be familiar with it, so that they can collaborate on research when LaTeX is used.

primary LaTeX language references
pdflatex on the command line
Most LaTeX users use it like a compiler on the command line to convert .tex to .pdf format. Building LaTeX documents used to be messier, with a separate compile and link step (!) but now it is one-step unless you are messing with fancy bibliographies shared across multiple papers
Overleaf
For some projects, a WYSIWYG web-based client works OK. Massive howto wikis
Bibtex
if you expect to be writing many papers in a given area, the bibliography-management tool bibtex may be useful

How does one learn to do CS research?

apprenticeship
watching a Master
from peers
observing other grad students; adopt best practices and avoid mistakes
by example...from the literature
read (many) papers, do as they do. Don't underestimate imitation.
by extension
read (many) papers, extend a result found there

Research Question(s)

lecture #6 began here

Types of Research / Types of Research Papers

Avoiding Mistakes in Grad School

There are infinite possible mistakes. But perhaps you can avoid some of these ones.

personal experience; things I've witnessed
skipping the difficult part
do worthwhile research before you try to publish it. Usually worthwhile research is extremely difficult!
skipping the painful part
rejection is so painful, some would avoid publishing. If you don't want to publish, a lot, research is not for you!
letting your personal life ruin your studies/career
like when a student's fiance cheated on them while they were writing their dissertation
rabbit holes
it is actually kind of arrogant to enroll in foreign language class while you are locked in a deathmatch with your thesis/dissertation
moonlighting
it is actually kind of arrogant to take a side job to boost your income while you are being paid to be locked in a deathmatch with your thesis/dissertation
checking out early
common PhD student problem
being overly-stressed
Like when I had a grad student sort of blow up at his own defense

lecture #7 began here

Lecture #7 was Dr. Poutkaradze's talk.

lecture #8 began here

Reminder: Friday, 5min. talks

Impressions from Monday's Talk

What were the strengths and weaknesses of the talk?
What was the research topic?
What were the research questions/results/contributions/evaluation?

Avoiding Mistakes in Grad School (cont'd)

USU
not planning your time
possibly to half-hour or 15-minute increments; set aside more time than you think will be needed for things; but time management has to be efficient or it will take too much time! (over-management)
not budgeting
this includes: not going into foolhardy debt, not allowing for how much time and money it will take, doing degrees that won't pay for themselves
(im)perfectionism
Low standards is bad. Excessively high standards is bad. "Real-time Perfectionism" - achieve the highest level of excellence that you can within your resource constraints
thoughtco,
thinking like an undergrad
focusing on grades
goal is not passing or getting A's, it is mastery in your field
failing to plan ahead
(over/under) time management
being unaware of department politics
OK, so what do you know about our department politics?
not fostering relationships with faculty
so, what relationships with faculty should grad students "foster"
ignoring peers
not putting in face time
despite covid, or adapted to the requirements of covid, successful graduate students are present in the department and its activities.
forgetting to have fun
if you don't have ways to rest and de-stress, that can be bad
cbsnews

Avoiding Mistakes in Research

Four Steps starting research from scratch (CoR prolog prior to Chapter 3)

  1. Find a topic that's specific enough to be solvable in time available
  2. Question the topic until you find questions that catch your interest
  3. Determine the kind of "evidence" that readers will require in order to accept an answer to these questions.
  4. Determine whether you can find this evidence.

lecture #9 began here

Announcements

The rest of Lecture #9 is student oral presentations (HW#1 Part 1).

lecture #10 began here

Avoidable Mistakes in HW#1

I recommend that you not make one of these mistakes.
didn't write a literature survey
if you write an overview of the (various subareas of the) topic without tieing it to the literature, you don't meet requirements
didn't write a critique of a literature survey
critique need to drill into the sources of the survey being critiqued, and assess their apparent degree of analysis of their sources.
scope too broad, or scope too narrow
If you include extraneous work, or miss major pieces of the literature you aren't meeting requirements.
No organization
Many literature surveys are presented in taxonomy form. Yours doesn't have to be, but give some thought to how to structure the literature that you identify as being in-scope.
used "I", or the royal "we", or other personality distractions
In technical writing the goal is not to provide color.
Major format distractions
Technical documents should be gorgeous. All the reader should see is your content. The reader should not notice anything funny about your line breaks or stuff sticking out in margins, widows/orphans, etc.
Didn't learn how to type quotes in LaTeX
Two backquotes at the front, two apostrophes at the back
Referenced reference [1] a hundred times
If you are doing a critique of [1], spell out the details of [1] and why it is worthy in glorious detail at the front of your paper, and then refer to it appropriately. BTW if you did not like the survey you critiqued, it is time to either add a lot of value to your critique to make up for the survey's deficiencies, pick a different survey paper, or just write your own.
Incorrect/inconsistent capitalization
Generally, things get capitalized when they are a name of something. Fig. 2 is capitalized because Fig. 2 is the name of figure 2; same applies to table names. Is "Big Data" the name of something? Gray area, but usually not. And "Big data" is almost always wrong.
Incorrect grammar
Most common problems are number agreement, missing or extra determiners, mishandling of mass nouns, etc.
Incorrect punctuation. A space is required after (the end of) a sentence. Commas do not end sentences.
Failure to cite
If you borrow a figure from a source, you had better cite from where.
Major Bogons
If your turned-in-for-credit paper has text left over from the ACM template, or subsection(s) with no content at all, or ...
All-tertiary sources
If > 50% of your sources are tertiary, you are doing a high school report, not a graduate school literature survey.
Incomplete biblio information
Missing publisher, city, (for journals:) volume and number, year, etc. Exception: If the name of the thing includes the publisher and I recognize that name (e.g. ACM, IEEE)
Bogus page layout
Single-column = direct violation of spec. Double column with humongous margins = possibly unintentional violation of spec.

From Topics to Questions (CoR Chapter 3)

Did this chapter add to what you've already known about in terms of finding a research topic?

First some terminology:

subject
broad area of knowledge
interest
whatever you want to do. but know why you are interested in it, and why others might be interested in it also. Don't pick a topic just because a teacher is interested in it*. Don't pick a topic just to look cool.
topic
A starting place for your research. Not just narrowing scope, but taking an approach that asks a question whose answer solves a problem that readers will care about
bundling of topic and research question is far from universally accepted.
requiring a research question to which readers will care about the solution is more widely agreed upon.
heed the good advice about narrowing scope
a topic is probably too broad if <= 5 words. A topic may be too narrow if you can't find any literature on it.*
Note the dubious advice on selecting advanced project topic in 3.1.3
"Find what interests other researchers" - what are the pros and cons of following this advice?
question versus problem
many questions are not problems. Booth's idea of a problem is a question for whom not knowing the answer prevents us from knowing something else. Can you give a better definition of a research problem?
Note the suggestion to cherry pick in section 3.3
Pick a research question, decide on the answer, and then select facts to support your answer is exactly how you land a job at a trash media outlet. What are scientists supposed to do?
Build an inventory of [research] questions about your topic
Don't assume that the first research question you ask is the best one.
Sources of [research] questions
Ask questions:
  1. history of your topic
  2. structure and composition
  3. categorization
  4. what if you negate a question?
  5. what if?
  6. suggested by other researchers
Specific suggestions in this list of sources of questions [from 3.3.*] may not apply, but some of them do, or are directly related to common approaches in CS research. Which of these sound like CS question-types to you? What CS question types are not on this list?

In any case, making a list of [potential research] questions conveys certain advantages. Like what?










You can evaluate, prioritize, identify dependencies...

So What?
[Booth] calls this the most important question. How are they using it? Motivating a particular research can be crucial, e.g. in grant proposals, or to get industry to green light a project. Booth suggests a 3 step process:
  1. name topic
  2. add an indirect question
  3. motivate topic by answering indirect question

From Questions to a Problem (CoR Chapter 4)

Suggested madlib for researchers:

                                                                 Evolution
Topic: I am studying _______________,                    <--- data collector
   Question: because I want to find out _______________  <--- researcher
      Significance: in order to __________________.      <--- member of the
                                                              research community

Connecting Practical to Conceptual

  vs.  

Practical problems: condition=>cost

Conceptual Problems

Failure understanding this=> failure on something else that's bigger.

lecture #11 began here

Lecture #11 was the seminar given by Dr. Kenny Gruchalla.

lecture #12 began here

Any feedback for me on HW?

Friday's class: Zoom

I will be in Colorado on Friday. I will attempt to hold class at the appointed hour, via zoom, but if I have internet challenges, there'll be no class.

Reading

Your required reading this week extends to Craft of Research Chapter 5.

Rest of class: we Crashed UIdaho's Colloquium

After a simple vote, we crashed UIdaho's colloquium, a talk on a web application that assists medical students to practice.

lecture #13 began here

Wednesday's colloquium: Dr. Giri Krishnan

Time to learn the value of sleep.

lecture #14 began here

Colloquium Reflection

Dr. Derrick Phillips

What were the strengths and weaknesses of the talk?
What was the research topic?
What were the research questions/results/contributions/evaluation?

Dr. Giri Krishnan

What were the strengths and weaknesses of the talk?
What was the research topic?
What were the research questions/results/contributions/evaluation?

Preliminary Comments on HW#1

From submissions received by Friday AM:

CoR Chapter 4, cont'd

Pure vs. Applied

Pure aka Basic
addresses a conceptual problem w/ no direct bearing on the world
Applied
addresses conceptual problem that has a direct bearing on the world

Finding a good research problem

Methodology:
  1. ask for help
  2. read...and look for problems in what you read
  3. question (your own and others') conclusions

Working with Problems

Managing your inexperience

Problems to Sources (Craft of Research Ch 5)

Three types of sources:
primary
original materials, draw data, from the creators/discoverers themselves
secondary
books, articles, reports that quote from or are based on primary sources. "the literature". Often peer-reviewed. But don't use secondary sources when primary sources are available.
tertiary
synthesis of materials culled from secondary sources. Wikipedia articles. Trade magazines. Things that are not peer-reviewed. arXive. Citing tertiary sources marks you as a novice, and many readers won't take you seriously. It is not just about did the source give you the information, it is about whether you can be believed when you use it.

Navigating the 21st Century Library

Great for historians...what about computer scientists?

because so much information is now at our fingertips, libraries are more essential than ever









Bias confession: I love libraries. But what are the downsides with libraries, and what are the alternatives for computer scientists?

...Back to CoR Ch 5

Some options include:

personal library
both a physical notion and a mental one. Compare with the notion of a "Personal Health Library". You are building a physical or digital collection of assets to which you have immediate access. What are the pros and cons of this just being a page full of hyperlinks?
advanced google ninja skills...and knowing what else to search besides just google
I have met a lot of students whose google fu doesn't cut it. What does it take to kind something on the internet that's not in the first google results from your first search for it?
knowing who knows
The Fahrenheit 451 model: be a social animal and cultivate relationships with people who can help you. Sometimes this is a librarian. Sometimes it is the right librarian. Sometimes it is a more advanced student, or a faculty member. Sometimes it is someone whose colloquium talk that you heard was on a related topic.
What are the online CS forms of "systematic and productive methods for discovering useful and credible sources"

lecture #15 began here

HW#1 Update

Reference Works and Online Databases

What are the best CS references?



I suspect that subdomains and various CS research communities have references of their own. Are there any with broad credibility?

Finding Specific Sources

Locating sources on the internet

Most of what we find using google is perfectly reliable, but not everything is.
Where else do you look? What else should you do/use?











You have to develop ways of filtering. I have known good sources, known bad sources, and everything else is suspect. They have been accrued by experience.
Known Good sources:
IEEE, ACM, Springer, Elsevier, Taylor & Francis, ...
Known Bad sources
predatory publishers; vanity conferences, including famous ones
Suspect
Certain publishers like Hindawi or MDPI have (or had, in the past) famously low standards, and can actually be a liability on your CV, instead of an asset. Conferences that have too many people on the conference program, and conferences whose scope is all of CS and beyond, are ones that might not provide peer review by experts in your field.

Evaluating Sources for Relevance and Reliability

relevance
skim, check the intro, conclusions, bibliography, etc.
reliability
is the publisher credible? is the work peer reviewed? is the author credible? how good is the bibliography? is the work obviously biased? how has it been reviewed/received? is the work cited by others?

Looking Beyond Predictable Sources

Using People as Sources

[Booth] means this in more of a social science sense, but as a computer scientist, we can twist this a little bit and note that many of the primary authors of work we care about in our research are still around, and many are actually happy to help, if asked nicely enough. Going to conferences can help a Lot in this business.

Amusing "Primary Source" Example

lecture #16 began here

HW#2

I decided after all to post you a one-week HW#2 that will improve on your literature survey efforts.

Amusing "arXiv is not a panacea" Story

For those of you who were citing many references from arXiv:

Engaging Sources (CoR Chapter 6)

Recording COMPLETE Bibliographic Information

Nothing is more frustrating than having the perfect quotation of bit of data in your notes, but not being able to use it in your writing because you didn't completely document your source, and can't find it again. - [Booth et al, p. 89]

Engaging Sources Actively

Yet another set of instructions on how to read a research paper. Writers frequently report that their work has been misstated, misquoted, misinterpreted... don't let that pass in your sources, and don't propagate it.

Reading for a Problem

Reading for Arguments (CoR)

Mimic/follow the patterns of good arguments and lines of reasoning that you find in your peers' research. This is a form of adopting best practices. "You do not plagiarize a source when you borrow its ways of arguing or of analyzing data". While you don't strictly have to cite in this case, you are usually better off if you give credit. It may strengthen your own credibility.

Reading for Data and Support

Taking Notes

This section might be recommendations for history students. But it applies to that smaller part of our job where we are responsible for citing related work. We do read a lot of secondary sources, and we probably take worse notes than historians.
paper notes...on index cards?
LOL, we won two world wars with that level of notetaking Using paper forces you to think about what's most important.
Using notecards let's you rearrange
Playing with spatial layout reveals connections
.docx file per source?
cumbersome indexing/sorting/searching
Fancy notetaking app?
which ones have you used, and would you recommend?
Reference Management System
Fancy database / CMS
Quote, Paraphrase, or Summarize
Most of my related work sections are summaries and comparisons. Under what circumstances in CS work is a quote (or paraphrase) appropriate?
Context
Notes need enough context to understand later and avoid misinterpretation

Annotating Your Sources

What are lightweight equivalents for writing in the margins of a book or article?

lecture #17 began here

Managing Panic from too much Information

Making Good Arguments (CoR Ch. 7)

We continue with fluffy barely-applicable-to-CS materials of [Booth etal].

Arguments as a conversation

Supporting your claims

claim
assertion/hypothesis/thesis
reasons
this is the "because X" answer why your claim is true
evidence
data in support of reason(s)
CS folks do this (support claims) in our research papers; how do we do this?

Acknowledging and Responding to questions and objections

Warrants

In general, this does not come up in CS much. It will come up when research is sketchy or the reader/reviewer/referee is less familiar with the subject matter. Basically a warrant is a big detailed explanation of why the reason implies the claim.

Warrant
Not as used in colloquial English; in this chapter a warrant is:
There are a lot of things that can go wrong here.

lecture #18 began here

Lecture #18 was colloquium given by Dr. Afsah Anwar.

lecture #19 began here

Warrants: more to come

We left off last Friday with an incomplete discussion of "warrants" in technical writing. If you are not yet comfortable with warrants, have no fear, there is a whole chapter in CoR on that topic that we will get to soon.

Impressions from Monday's Talk

What were the strengths and weaknesses of the talk?
What was the research topic?
What were the research questions/results/contributions/evaluation?











Composing a Complex Argument

The better you understand the subject, the better you will write about it.

Thickening Your Argument

your ethos
your "character" as projected by your argument

Avoid falling back on what you know

confirmation bias
the tendency to interpret new evidence as confirmation of one's existing beliefs or theories.

Making Claims (CoR Chapter 8)

Kinds of Claims

Usually, conceptual rather than practical.

Give a CS example (if possible) for each of these kinds of claims:

Fact or existence
There exists a O(n) sort algorithm
Definition or classification
...
Cause and consequence
...
Evaluation or appraisal
...
Action or policy
...

Practical Claims

Practical claims argue for or against some action or policy.

Evaluating Claims

How specific are they? How significant are they?

Specificity of claims

Acknowledge three kinds of different views:
  1. Readers' beliefs that your claim challenges
  2. Conflicting viewpoints
  3. A condition that limits the scope of your claim
Why all this acknowleging of conflicting views:

An aside into "Mammalian brain"

I was once taught how to be respectfully contrary by a Very Expensive outside consultant. A little bit of that is relevant here: Although to some extent this should not apply to technical writing in computer science, to some extent it does.

A few thoughts from Chapter 1 of Ramage and Bean

I am not going to require you to go buy this text, but here are some tidibits from Writing Arguments, by John Ramage and John Bean, published by Allyn & Bacon, Needham Heights MA, in many editions since at least 1999. If you go into the research business, you might want to check this book out.
argument combines truth seeking and persuasion
an author should write as much with the audience in mind, as the subject they are writing about!?
Socrates versus "Darth" Callicles
Socrates is pursuing absolute truths, which Callicles scoffs at. Socrates doesn't win... Callicles' strategy is to not allow Socrates any assumptions. Skeptical science readers and paper referees occasionally do the same.

Significance of claims (from CoR chapter 8)

Qualifying claims

Assembling Reasons and Evidence (CoR Chapter 9)

Reasons

How do you distinguish reasons from evidence?

lecture #20 began here

Lecture #20 (10/2/23) will be Dr. Hasan Jamil, hosted by the university of Idaho.

lecture #21 began here

Impressions from Monday's Talk

What were the strengths and weaknesses of the talk?
What was the research topic?
What were the research questions/results/contributions/evaluation?

Evidence

Evidence vs. Reports

Evaluating Evidence

If you know how to evaluate your own and others' evidence, you also know how you will be evaluated.
accurate
even the most trivial mistake will be used to dismiss your claim outright or cast your whole argument into doubt. If you can't capitalize or use punctuation correctly, I am hardly going to trust your claims about computer software performance.
precise
in CS we often talk about digits of precision, and the most common claim is that the number of digits of precision given exceeds the precision of the actual measurement.
sufficient
Is the evidence sufficient to support the claim? Beginners' evidences are usually insufficient.
representative
you can't "cherry-pick" your data. sampling data is error prone and needs to be shown to be representative. anecdotal evidence might or might not be representative
authoritative
best way to learn is by failed examples. Ironic to use CDC vs. Wikipedia as examples of authoritative vs. not

HW#3 Reminder and Discussion

lecture #22 began here

On Case Studies (part 1)

We covered slides 1-?? or so.

lecture #23 began here

Talk by Marin Talipov

lecture #24 began here

No Class Friday

Let's not forget that the VPAA has declared Friday an academic holiday in honor of "49ers".

Grading

I've been working on grading your HW#2's. They will be graded and figured into the midterm grade by Monday.

Acknowledgements and Responses (CoR Chapter 10)

Ways readers might question your argument:
Intrinsic soundness
questioning your claim, reasons, and evidence
Extrinsic soundness
alternative ways to frame the problem, things you've omitted

Questioning your argument like a reader would

Imagining Alternatives

Deciding What to Acknowledge

Priorities:
  1. predictable "weaknesses" that you can rebut
  2. common alternative views/arguments/beliefs held in your field
  3. other conclusions readers would prefer be true
  4. alternative evidence readers may be familiar with
  5. important/apparent counterexamples

Acknowledging your Flaws

lecture #25 began here

HW#3 Presentations

lecture #26 began here

HW#3 Presentations

lecture #27 began here

Monday preview

HW#4 Discussion

Questions you can't answer

Responses as subordinate arguments

Vocabulary of Acknowledgement and Response

Decide how much credence to give each objection
downplay: "despite/regardless/notwithstanding/although/while/even though"
when an objection is valid, but doesn't change your ground truth
"seem/appear"
when an objection is apparent, but is not really so
"think/imagine/say/claim"
to dismiss hypothetical objections from nameless sources
"some/many/a few"
to generic but nonimportant real-world objections
"understand/know/realize"
you can own up to a valid objection without downplaying it. it has validity even if it doesn't change your mind, overall

Responding to Objections

You may need to refute objectioners' evidence with blunt logic, or with tact and deference, depending on your relationship to them.

3 Predictable Objections

  1. there are additional/alternative causes to the one you claim
  2. what about these counterexamples?
  3. I don't define X as you do. To me X really means...

Warrants (CoR Chapter 11)

warrant
a general principle that connects a reason to a claim
State warrants when Clint's take:

Warrants in Everyday Reasoning

proverb
a warrant that we all know
If your reader doesn't already know and accept your warrant, its not a warrant. A warrant connects a circumstance to a consequence.

Warrants in Academic Arguments

Academic warrants aren't commonplaces we all share
They are principles shared within a community of researchers
Experienced researchers rarely state their warrants
Novices have to learn the logic used by the community of researchers -- what flies and what doesn't.
The text goes into an example warrant: a whale is closer to a hippo than to a cow, because its DNA is more similar. Biologists do not have to warrant that DNA similarity is a way to measure species relationship.

lecture #28 began here

Lecture #28 was a talk by David Torres.

lecture #29 began here

Colloquium Reflection

Dr. David Torres

What were the strengths and weaknesses of the talk?
What was the research topic?
What were the research questions/results/contributions/evaluation?

New Assignments

Be sure to note the new assignments, including one due Friday 11/3:

For this assignment, you are to write a ~5 page paper discussing the articles below, which are probably largely redundant. Alternatively, if you actively participate in the class discussion on this topic on Friday 10/27/31, you can write and submit a one paragraph summary of that discussion and receive full credit.

The topics you are to explore include: (a) if you attended the SWE 2023 conference, to what extent the event resembled (or did not resemble) what is described to have happened at the recent Grace Hopper Conference. (b) What alternative events would be available to persons who are not members of the selected underrepresented group in order for them to network and apply for jobs with big tech companies. (c) What are the ethical and business implications for the Big Tech firms, if the best or only place for individuals to network and apply for jobs with big tech are at conferences for under-represented groups? (d) What sort of behavior would be expected and appropriate for a non-member of an under-represented group that is attending a conference that is intended for members of an under-represented group?

For Friday, read these articles and take notes.

Monday's talk: Nirmal Budhathoki

Something for you to look forward to...we will join Idaho's Grad Seminar zoom.

Logic of Warrants

Specifics are true because of general principles.

Testing Warrants

  1. Is the warrant reasonable?
  2. Is it sufficiently limited?
  3. Is it superior to any competing warrants?
  4. Is it appropriate to the field of study?
  5. Does it cover the reason and claim?

When to State a Warrant

your readers are outside your field
they do not know your field's warrants
when you use a principle that is new or controversial
warrants have to start somewhere
readers do not want your claim to be true
and need to be told why they should accept it

Testing Warrants

The example warrant was in support of a claim that videogames make children more violent.

The warrant that when children are exposed to violent images, they are influenced for the worst.

The warrant is true but does not tie exposure to violent images with a rise in youth violence.

Warrants and Proverbs and Principles in Computer Science

From googling, here and here and here and here and elsewhere.

Consider these to be "warrant candidates". Which ones are actual warrants? i.e. Which ones involve an underlying principle we would agree with and share in common?

80-20 rules
In CS there are many, including "programs spent 80% of their wall clock time execution 20% of their code".
Computer science is creating the right model for a problem... and devising the appropriate techniques to solve it.
A. Aho
If you haven't figured it out yet, use a brute force algorithm
Ken Thompson
The faster the code is written, the slower the program will run
Roy Carlson
If the code and comments are inconsistent, it is likely that both are wrong.
Norm Shryer
If there are too many special circumstances, you must be using the wrong method
Craig Zerouni

lecture #30 began here

Discussion of Crashing CS Conferences for Underrepresented Groups

What is the Grace Hopper Celebration?
Who is it for?
What is the context of this year's GHC?
What are your thoughts on the following quote?
You (cisgender men) thought you could come here and take space to try to get a job.
What are your thoughts on the following quote?
I was forced to leave the queue due to unchecked men displaying aggressive behavior, ganging up to trample women.
What are your thoughts on the following quote?
Some of you took discount academic tickets ... and interview slots from women who need them. All of those are limited resources to which you have no right.
What alternative events are available to persons who are not members of the selected under-represented group in order for them to network and apply for jobs with Big Tech firms?
What are the ethical and business implications for the Big Tech firms, if the best or only place for individuals to network and apply for jobs with them are at conferences for under-represented groups?
What sort of behavior would be expected and appropriate for a non-member of an under-represented group that attends a conference intended for members of an under-represented group?

lecture #31 began here

"Warrant Candidates" in CS (cont'd)

As a reminder, these are only warrants of most or all of us would pretty much agree that we know them and agree with them as CS principles.
First figure out the data structure, and the rest of the program is self-explanatory.
David Jones
Bad programmers worry about the code. Good programmers worry about data structures and their relationships.
L. Torvalds
Make the user interface as consistent and predictable as possible
Principle of least surprise
Don't let users provide information that the system already knows
Rick Lemons
A test can only prove that a program has errors, but not that a program has no errors.
E. Dijkstra
If things are not broken, don't fix them
R. Reagan
The first step in fixing an error is to reproduce the error.
T. Duff
On a non-I/O intensive program, more than half the runtime is spent on less than 4% of the code
D. Knuth
Do things in the simplest and dumbest way
KISS Principle
90% of the functionality delivered now is better than 100% delivered never
Adding human resources to a late project makes it later.
Brooks's Law
Implement things when you actually need them, not when you "foresee" that you will need them.
To revisit "every problem in CS can be solved by yet another level of indirection."
B. Stroustrup, or B. Lampson. A professional colleague endorsed it saying "I've put it to good use many times in the form of proxies and edge clouds. And of course, you see it used in many areas of CS: indirect memory addressing in hardware, pointers, DNS systems, pub-sub systems, proxies, etc."
Corollary to the preceding warrant: "There is no performance problem that cannot be solved by eliminating a level of indirection."
Jim Gray.
It always takes longer than you expect, even when you take into account Hofstadter's Law".
This is Hofstadter's Law.
An evolving system increases its complexity unless work is done to reduce it.
Software systems do not work well until they have been used, and have failed repeatedly, in real applications.
D. Parnas
Before software can be reusable it first as to be usable.
R. Johnson
Complexity is where the bugs hide.
Controlling complexity is the essence of computer programming
B. Kernighan
There are two ways of constructing a software design: One way is to make it so simple that there are obviously no deficiencies, and the other way is to make it so complicated that there are no obvious deficiencies. The first method is far more difficult.
Tony Hoare
Computer Science is the only discipline in which we view adding a new wing to a building as being maintenance.
Jim Horning
All the important problems in computer science have to do with scalability.
when debugging, novices insert corrective code; experts remove defective code.
Richard Pattis

lecture #32 began here

Software engineering is that part of computer science which is too difficult for the computer scientist.
Friedrich Bauer
Debugging is twice as hard as writing the code in the first place. Therefore if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it.
Brian Kernighan
Easy to use and hard to misuse
Everything should be made as simple as possible, but no simpler.
A. Einstein
Functions should do only one thing, well. (cohesion)
In programming the hard part isn't solving problems, but deciding what problems to solve.
P. Graham
Dependency hygiene trumps code reuse
It can be better to duplicate a little code than to pull in a big library for one function.
Focus comes from saying no to 1,000 things to make sure we don't get on the wrong track or try to do too much.
S. Jobs
It is not uncommon for the cost of an abstraction to outweigh the benefit.
J. Carmack
The three great virtues of a programmer are laziness, impatience, and hubris.
L. Wall
Managing programmers is like herding cats.
More computing sins are committed in the name of efficiency than for any other reason, including blind stupidity.
W. Wulf
The biggest problems in software are problems of misconception.
R. Hickey
Nine people can't make a baby in a month
F. Brooks
A specification that will not fit on one page of 8.5x11 paper cannot be understood
One Page Principle, M. Ardis
Program to an interface, not to an implementation.

lecture #33 began here

Lecture #33 was a talk by Dr. Jun Zheng.

lecture #34 began here

Colloquium Reflection

Dr. Jun Zheng

What were the strengths and weaknesses of the talk?
What was the research topic?
What were the research questions/results/contributions/evaluation?

To Extend or Not to Extend

"Warrant Candidates" in CS (cont'd)

Simplicity is prerequisite for reliability.
E. Dijstra
Simplicity, carried to the extreme, becomes elegance.
J. Franklin
A program is never less than 90% complete, and never more than 95% complete
T. Baker
Trees sprout up just about everywhere in computer science.
D. Knuth
The fastest algorithm can frequently be replaced by one that is almost as fast and much easier to understand.
D. Jones
The first 90% of the code accounts for the first 90% of development time; the remaining 10% accounts for the other 90% of development time.
T. Cargill
The first rule of functions is that they should be small.
Robert Martin
The purpose of software engineering is to control complexity, not to create it
P. Zave
There are only two hard things in computer science: cache invalidation and naming things, and off-by-one-errors
P. Karlton, L. Bambrick
To understand recursion, you must first understand recursion
You can't trust code that you did not totally create yourself.
K. Thompson
Designs that just work are much more difficult to produce than designs that are just assemble long lists of features
D. Crockford
Premature optimization is the root of all evil
D. Knuth
Well-designed components are easy to replace. Eventually, the will be replaced by ones that are not so easy to replace.
Sustrik's Law

A few more CS warrant candidates

I will skim through these pretty fast. Again, they are only useful warrants if they identify some principle that most computer scientists would agree, and to which we therefore could appeal in support of our reasons for a claim.
The number of transistors doubles every two years. The cost of computers is halved.
Moore's Law.
Software is getting slower more rapidly than hardware is getting faster.
Wirth's Law
Roughly every decade a new, lower priced computer class forms based on a new programming platform, network, and interface resulting in new usage and the establishment of a new industry.
Bell's Law of Computer Classes
Given enough eyeballs, all bugs are shallow
L. Torvalds
The value of a network is proportional to the square of the number of connected nodes.
Metcalfe's Law
The value of a network scales exponentially with the number of connected nodes.
Reed's Law
New interconnections introduce new vulnerabilities.
A security corollary to the software engineering antipattern known as "high coupling"
The complexities of computerized systems cause new insecurities.
insecurity is one form of bugginess.
The major difference between a thing that might go wrong and a thing that cannot possibly go wrong is that when a thing that cannot possibly go wrong goes wrong it usually turns out to be impossible to get at or repair.
Pessimism (see Murphy's Law) demands planning for all possible failures
Most software is poorly written and insecure.
Pessism, applied to one's understanding of one's fellow human beings
The extensibility of computing systems enables their weaponization
Does anyone remember back when Microsoft OLE was a thing? Word Macros ... led to Word Macro Viruses
Garbage in / Garbage out
Basic principle of computing. If the input data is invalid, the results of a computation are meaningless
The best performance improvement is the transition from the non-working state to the working state.
J. Ousterhout
Avoid multiple maintenance
Refactor common code in multiple places to where there is only one copy of the code in one place where fixes may be applied.
Preserve locality of reference
A principle that applies to both data and to code.
Plan to throw one away
F. Brooks formulated the argument for prototyping
The general tendency is to overdesign the second system.
also F. Brooks

lecture #35 began here

HW Extension

I haven't heard from folks wanting to present their improved research proposal, so we will extend the due date slightly, to 3pm on 11/17.

Planning and Drafting (CoR Chapter 12)

The tendency of the CoR book to reduce a complex topic to a simple formula for you to follow, is both a strength and a weakness.

Sketching a working Introduction

"Plan to throw one away." - Fred Brooks
Sketch your working introduction to address three parts.
write a brief summary
only key points and research/sources your paper challenges, modifies, or expands upon.
rephrase your research question as a statement about a flaw or gap in existing sources/knowledge
Awkward semi-example: "Can we build a better debugger atop a high level execution monitoring framework?" becomes "The rate of advances in debugging will increase if we make it easier to write debuggers."
Sketch an answer to "So what if we don't find out?"
The obvious purpose here is to motivate the importance of this work
State the answer to your research question as the main point of the paper
This may go at the end of the intro, or in the conclusions.

Plan the Body of Your Paper

This smells halfway in between (a) traditional term-paper-writing instructions that work from an outline and gradually flesh things out until the paper text is derived, and (b) instructions on how to develop an object-oriented design, writing a page/card for each class, what the classes' responsibilities are, and sorting out how they relate to each other.
  1. Sketch background and define terms
  2. Create a page for each major section. what does it support, develop, or explain?
  3. Find a suitable order. Sometimes the order that makes sense to you might not meet the needs of your readers. The most common issue is whether the related work needs to be presented early, or late.

Plan Sections and Subsections

This is fleshing out the contents of the "page" you created for each section. It is a storyboard or an action plan for what you are going to write.
highlight key terms
each section needs its own little introduction
indicate where to put evidence/acknowledgements/warrants/summaries

Sketch a Working Conclusion

Be sure your paper can answer the question: "so what?". The reader has to be able to see the significance of your result. Why was your paper worth reading? Why was the research work worth doing?

Avoid 3 Common Flaws in Planning

The paper must not be a narrative of your thinking.
This isn't the history/log of your work, and it doesn't need to show all your dead-ends. If any discarded attempts are discussed at all, it might be to acknowledge and refute plausible sounding approaches that were attempted and found not to work. Too much "I" in your paper is a red flag.
Do not assemble your paper as a frankenstein concatenation of your sources.
A literature review paper is almost that, sometimes. But readers want value-added, not just stuff other papers say. You generally need to add organization, analysis and/or new solutions not found in sources.
Do not map your paper directly onto the language of your assignment!?
Wow, really. Echolalia is not actually a positive. If your paper is for an assignment, do not write the text of the assignment or its rubric! Instead, meet the objectives stated, and satisfy the rubric, with your own direct language. One interesting possible contradiction of this is that I've seen plenty of NSF program announcements where the proposal has to literally say specific things specified in the announcement or it will not be considered.

Turning your Plan into a Draft

lecture #36 began here

Avoiding Procrastination and Writer's Block

If you feel too intimidated by what you are trying to do
Break up your big task into smaller tasks until you are able to proceed
If you have no (or unreasonable) goals
create a routine that sets small/reasonable goals. Use a progress chart or meet with a partner to force progress.
Your sentences/paragraphs are never good enough
Write informally "to get the ideas on paper". Compromise on perfection in order to get the job done.
There are in most institutions places like a Writing Center where you can get expert help. Use them!

Organizing Your Argument (CoR Chapter 13)

"We all know our own work too well to read it as others will."

Thinking Like a Reader

Revising your Frame

"Frame" denotes what the readers must be able to find instantly:
  1. (If your paper format allows it, CoR authors say to literally) put an extra vertical space in after the intro and before the conclusion.
  2. State your main point shortly before the end of your introduction.
  3. Your main point sentence should contain key terms that identify the concepts and themes of your paper.

Revising your argument

identify the substance
is it focused? does it stay on topic? There is also a right balance between reasons and evidence.
evaluate the quality
reliable? qualified/limited? conversational? warranted (if necessary)?

Revising the Organization of your paper

Checking Your Paragraphs

Letting your Draft Cool

Abstract

State the research problem, announce the key themes, and state the main point.

Incorporating Sources (CoR Chapter 14)

Quoting, Paraphrasing, and Summarizing Appropriately

Integrating Direct Quotations (CoR Chapter 14, cont'd)

Showing Readers How Evidence is Relevant

lecture #37 began here

One more talk for the semester

Dr. Roopa Vishwanathan of NMSU has agreed to give a talk via Zoom on the Wednesday after Thanksgiving.

Zero more homeworks for the semester

...Back to CoR, Chapter 14

The Social Importance of Citing Sources

Common Citation Styles

CS (ACM, IEEE, etc.) tends to cite things differently than in the humanities and social sciences. CoR describes four methods:
author-title styles
author-date styles
We pretty much don't use any of these in CS. We often use [1] style, numbered from the order they appear in the text. I have also seen and used commonly a style consisting of abbreviated author and year, as in [Jeff21]. Mostly, where you submit specifies how you cite, and you should expect to have to follow it exactly, and that different styles will be required in different venues.

Guarding Against Inadvertent Plagiarism

Communicating Evidence Visually (CoR Chapter 15)

When to use pictures

Choosing the Most Effective Graphic

Some Common Types of Graphics

Designing Tables, Charts and Graphs

Just for Fun: Minard's Napoleon's March

lecture #38 began here

Grading and End of Semester Stuff

Communicating Evidence Visually (cont'd)

Just in time for the "Napoleon" movie, we ended last class with probably the most famous graphic in history. Charles Minard's map of Napoleon's Russian campaign of 1812 is a 2D map of central/eastern europe, with troop strength, direction, dates, and temperature information, capturing 6-7 dimensions legibly on a single image, a remarkable feat.

Jon Snow's Cholera Map

As seen on TV and in a past NMT CSE colloquium talk by Stephen Kobourov, Jon Snow may know nothing, and yet in 1854, years before the development or adoption of germ theory, Jon Snow stopped a cholera epidemic by creating the following graphic and following the direction that it indicated.

A multi-dimensional line chart

Fisheye views

Data-ink minimization example

A Space Metaphor

3D Layouts

3D Trees

Round Call Stacks

Kiviat Diagrams

World Clouds

Gazebo Metaphor

Using Visualization in a Debugger: DDD

lecture #39 began here

How was the colloquium last Wednesday?

What were the strengths and weaknesses of the talk?
What were the research topics?
What were the research questions/results/contributions/evaluation?











Improved Research Proposal Comments and End of Semester

HW#4 Rubric

Name ___________________________________ 
Explicit research question(s)  ___ / 3
Related Work                   ___ / 4
Proposed Solution              ___ / 4
Depth                          ___ / 4
Acknowledge/Respond            ___ / 2
Budget                         ___ / 2
Format                         ___ / 1
---------------------------------------
Total                          ___ / 20

Introductions and Conclusions (CoR Chapter 16)

Common Structure of Introductions

contextualizing background
statement of the problem
response to the problem

Context

Define those states and processes that constitute the domain within which the problem will be encountered/observed. Context may include pointing out commonly-held misunderstandings and flaws in current thinking or previous results.

Stating the Problem

condition
incomplete knowledge or understanding, stated directly or indirectly
consequences
what is the cost, human or otherwise, of us not getting this? Or what is the benefit of us getting this?

Stating your response

After disrupting the readers' peace and happiness by stating a serious problem and its consequences, you have to either This is generally in the last few sentences of your introduction.

Setting the Right Pace

How fast you can go in your introduction depends on:
your audience...
how much they know; your common vocabulary
and your topic
how familiar is your context and/or your problem to these readers?

Organizing the introduction

Context + Problem + Response

with the following shortcuts/special cases:

formulaic == good

give your readers what they expect, and save your surprises for your results

Finding your first few words

Are the first few sentences the hardest? Avoid cliches. The desire for shared context or common ground does not give you an excuse to indulge in one of these. CoR researchers instead suggest three "standard" choices:
a striking fact relevant to your problem
a striking quotation
but only if it hits many key words in your domain
a relevant anecdote
but only if it vividly illustrates your problem
Yeah, none of these is great for a computer science research paper. We do not expect such parlor tricks. What computer scientists want to read is usually a no-nonsense direct statement of the domain.

Writing your Conclusion

Titles

Revising Style (CoR Chapter 17)

lecture #40 began here

lecture #41 began here

lecture #42 began here

Compare

A. Too precise a specification of information-processing requirements incurs a risk of a decision-maker's over- or underestimation, resulting in the inefficient use of costly resources. Too little precision in specifying needed processing capacity gives no indication with respect to the procurement of needed resources.
vs.
B. A person who makes decisions sometimes specifies what he needs to process information. He may do so too precisely. He may over- or underestimate the resources that he needs. When he does that, he may use costly resources inefficiently. He may also fail to be precise enough. He may not indicate which resources others should procure.
vs.
C. When a decision-maker specifies too precisely the resources he needs to process information, he may over- or underestimate them and thereby use costly resources inefficiently. But if he is not precise enough, he may not indicate which resources to procure.
What are your impressions?




Given all this, here are:

Two principles of clear writing

Distinguishing Impressions from Their Causes

Principle #1

Your sentence subjects should name the main characters in your story. Use short, specific, concrete subjects in your sentences.

Avoiding Nominalization

Principle #2

Express your crucial actions in verbs, as actions taken by your principle #1 subjects.

Two Tests for Diagnosing Problem Sentences

Underline the first 6-7 words in each sentence/clause, and then perform two tests:

How to Revise Problem Sentences (to be clear and direct)

Don't take this CoR advice literally for CS technical writing. It expresses some valid principles. Dr. J adds: break any long sentences that don't fit this direct (character-subject, action-verb) sentence pattern into smaller sentences that do.

Who or What can be a Character?

Creating Main Characters

Most stories have several characters to choose from. You turn them into main characters by making them the subject of multiple sentences.

Old before New

To diagnose: To revise:

Active and Passive Voice

Last Lecture

If we got this far, one of the things I'd want to share is Randy Pausch's Last Lecture