PhD candidate at @MILAMontreal studying deep learning and reinforcement learning. Formerly @jhuclsp, a @GoogleAI resident, and @SCSatCMU.

Joined December 2016
Permanent offer: if anyone wants a high-effort, public, non-anonymized, most-likely-critical review, please send a draft of your paper my way. I can't promise I will help you get into conferences, but I will do my best to help improve the quality of the science.
1
5
1
148
Great work from @frankstefansch1 and co-authors. But now comes the important part: what are the consequences for the dozens of researchers who, by claiming their new optimizer was an improvement, published incorrect findings?
Replying to @frankstefansch1
Perhaps the most important takeaway from our study is hidden in plain sight: The field is in danger of being drowned by noise. Despite years of research by many committed authors, we are often still stuck with "trial-and-error" when it comes to deep learning optimization.
Show this thread
2
3
0
15
Will there be retractions of papers? I doubt it. Most likely, the majority of authors will not even acknowledge this work, and continue to discuss their findings as though they were legitimate. Will there be reputational damage? That's up to you, the community.
1
0
0
1
If we don't take action, the result of all that fake science will simply be to advance the careers of those involved. Even this paper, *directly debunking their work*, will help them, just becoming one more citation towards their h-index. Is that the incentive structure we want?
3
0
0
4
As @josephdviviano once said: "the best part about writing a bad paper is all of the citations debunking you"
0
3
0
7
Jacob Buckman retweeted
Homeopathy is a pseudoscience because it doesn’t come from academia. If it did, it would be call “a scientific field with a replication crisis”.
1
3
0
26
Paper writing tip: no matter the topic, always remember to cite (1) a random paper by Hinton from the 80s and (2) capsule networks, both within the first two paragraphs. Reviewers will assume that the paper is by Geoff Hinton and give you a free accept!
6
14
2
338
Jacob Buckman retweeted
I have a question to anyone knows group theory. A group acts an a set X, partitioning it into orbits. Another group acts on those orbits, giving orbits of orbits. Under what conditions the orbits of orbits are isomorphic to a group action on X? mathoverflow.net/questions/3…
0
1
0
2
Jacob Buckman retweeted
"Does it catch fraud..? No" "Does it prevent..basic errors..? No" "Does it stop..nonsense..from being published? No" "Does it consistently chose papers worth publishing? No" The Absurdity of Peer Review What pandemic revealed about scientific publishing elemental.medium.com/the-abs…
57
503
52
1,578
Show this thread
Thanks to everyone who joined!
Curious why I think my own papers our bullshit? Come find out this Friday, June 4th, at 3pm EST for a special session of "Bullshit Papers of Our Own"! It will be hosted on Discord: discord.gg/rce9eUrtR7
Show this thread
2
0
0
9
Happening in 30 minutes!
Curious why I think my own papers our bullshit? Come find out this Friday, June 4th, at 3pm EST for a special session of "Bullshit Papers of Our Own"! It will be hosted on Discord: discord.gg/rce9eUrtR7
Show this thread
1
0
0
1
Curious why I think my own papers our bullshit? Come find out this Friday, June 4th, at 3pm EST for a special session of "Bullshit Papers of Our Own"! It will be hosted on Discord: discord.gg/rce9eUrtR7
8
4
2
39
Curious why I think my own papers our bullshit? Come find out this Friday, June 4th, at 3pm EST for a special session of "Bullshit Papers of Our Own"! It will be hosted on Discord: discord.gg/rce9eUrtR7
8
4
2
39
(Shoutout @savvyRL for the name!)
1
0
0
5
Curious why I think my own papers our bullshit? Come find out this Friday, June 4th, at 3pm EST for a special session of "Bullshit Papers of Our Own"! It will be hosted on Discord: discord.gg/rce9eUrtR7
8
4
2
39
I also hope to build this Discord channel into a space for insightful, critical discussion around AI research. I think the community could benefit for a space where the default assumption is skepticism, and harsh public critique is encouraged.
2
0
0
4
If that sort of environment sounds fun, or you think your ideas are rock-solid and want to test them in a crucible, or even if you just want to sit back and watch the sparks fly...come through and join us!
0
0
0
1
("Reproducibility" as defined in the narrow sense typically used in the ML community, e.g. as evidenced by this checklist: cs.mcgill.ca/~jpineau/Reprod…)
1
0
0
4
Show this thread
And finally, re: solutions -- the suggestion of fraud was tongue-in-cheek, the real goal of the call-to-action was to be provocative enough to draw attention to the issue, i.e. what you described as #2. I really like your idea on public review as well!
1
0
0
3
Show this thread