Replace Makefile with cross-platform scripts

[MattMS]

It would be nice to replace the Makefile with something that works on Windows and others.

I'm happy to do the work, but I just need to confirm what functionality is needed.

From what I can pick out:

• Build /index.html, with redirect to latest version of docs.
• Download specific version of Ramda.
• Run GitBook on specific version of Ramda.
• Run JSDoc on specific version of Ramda.
• Run Less compiler.

Please let me know anything I'm missing?

[CrossEye]

While I agree with the idea, this may be more controversial than it might appear. JustUseMake is the brainchild of core contributor @davidchambers.

I would certainly suggest that if we went this way that we didn't try to use Grunt, Gulp, Broccoli, or whatever the flavor of the month is, but instead build with NPM scripts. It seems to have become eminently reasonable to do this these days, and if anything is missing, we can write our own scripts for it to run in Node.

I will try to look at your list later in the day (meeting about to start), but at first glance it looks pretty complete.

[davidchambers]

I would certainly suggest that if we went this way that we didn't try to use Grunt, Gulp, Broccoli, or whatever the flavor of the month is, but instead build with NPM scripts.

👍

[MattMS]

No offence intended! 😅

👍 the use of npm scripts over a big build tool, especially since most of the tasks seem to just be simple commands anyway.

[MattMS]

Any further thoughts on this?

One option could be adding the npm scripts alongside the Makefile for a while to see if they work out.

[branneman]

👍 — I'd say a Gulp script is much better than Make.

Grunt/Gulp > Make
IMO, we're better off having the build tools in the same language as the library itself. This lowers the bar for contributing. Your users are used to JavaScript, Make is not the common tooling for JavaScript.

Grunt/Gulp > npm scripts
Grunt/Gulp is a very thin layer around nodejs, and npm scripts simply are plain nodejs. I'd prefer Grunt/Gulp since it is a standardised way of working with tasks. When you roll npm scripts you're coding it all yourself manually, that feels quite like the "not invented here syndrome". You're not getting the advantages of the tooling. I do say: the less gulp modules the better.

Lastly, Make and Gulp are actually both a bit declarative, and most people are familiar with the syntax. Rolling your own npm script often means losing this, or are you planning on building the build script with Ramda itself? ;)

Gulp > Grunt
Gulp over Grunt is mostly a no-brainer, nodejs streams are the better way to handle these kind of tasks, you get free concurrency with it, thus better performance without ugly tools like grunt-concurrent.

Gulp > Broccoli
Gulp's community is way bigger, and the advantages of Broccoli's syntactic sugar over Gulp's syntax are not that strong if you ask me. A tiny advantage with a much smaller community.

The question is, are the core contributors open to a PR replacing Make with Gulp?
//cc @buzzdecafe @CrossEye @davidchambers

[arcseldon]

Interestingly, I have been going in the other direction during my JS career to date.

Grunt -> Gulp -> NPM Scripts

And I am not alone in this thinking either - Why I left Grunt and Gulp for NPM Scripts.

Been refreshing to get rid of all the grunt / gulp verbosity and dependency overhead. And, I do say that after consideration to gulp which has been of great help on quite a few projects to date.

In fact, have recently been considering Make as the next step... (for projects where lots of tasks make NPM scripts cluttered anyhow).

Likewise, I have shifted from:

browserify -> webpack

Sure this is sucking eggs or else others have their own preferences and workflows that work best for them. Think it is much to do with what you are familiar with, and some subjectivity too.

Grunt/Gulp > Make - This lowers the bar for contributing.

Sounds correct, but is the Makefile really that difficult to understand (if you are already familiar with task / build tools in general) ?

[arcseldon]

With regards to Windows, did find this SOF answer - it certainly doesn't sound ideal but might be a workaround?

[arcseldon]

Ok, yep there is Platform specific instructions inside the make file

[arcseldon]

@MattMS - are you on Windows OS? Was wondering whether (potentially with some tweaks to the makefile) you could get it running with Cygwin and Nmake. Was hoping to check myself but my VirtualBox instance of Windows trial period has expired.

Either way, it appears there is a valid case to be made for cross-platform support, even if that only means providing a bat file / NMake compatible script.

[branneman]

And I am not alone in this thinking either - Why I left Grunt and Gulp for NPM Scripts.

Sure, but writing an article about something doesn't make it true :) I disagree wholeheartedly with that article. I think the real problem that the article should address, is overuse and reliance on (bad) gulp plugins.

But that is not a problem with Gulp, that a problem with how it's used. People tend to "know gulp", instead of really knowing how an OS or Node.js works. That's why there's an overuse of gulp plugins. That's why there's a movement that prefers npm scripts.

Take a look at vinyl-transform for example, if used well you'd need almost no gulp- prefixed npm modules at all. Your gulp script would even start to be quite like a plain nodejs script called via npm. But the gulp script would have more structure, you can take advantage of sane dependency handling, etc.

The fact that you're using Gulp doesn't mean you can't use npm plugins that don't have a gulp- prefix. Again, Gulp is just a very thin layer around nodejs, it's just javascript. You do have the power of all npm at your disposal. But that's not how the average gulp user writes it's build script.

When using npm scripts, for anything more complex than doing webpack -p you should write that functionality as separate node modules that your pre/post/regular npm scripts simply reference. At best, you're just trading the maintenance of gulp/grunt scripts for the maintenance of custom node modules.

Gulp is a standard for JavaScript users. Make is also a standard, but not for JavaScript users. Writing your own nodejs script and then putting it in your package.json is inventing your own way of working.

[arcseldon]

@branneman - I hear you, and you do make a lot of sense - much of this depends on context, and picking the right tool for the job, plus your own competency level and experience with what you like to use.

My own usage of NPM Scripts tends to be succinct statements placed inside package.json, and where possible using NPM packages like rimraf to omit cross-platform incompatibilities. In general, I write packages (both personal and commercially) that target *nix systems exclusively.

Given this is an open source project, with mixed audience in terms of knowledge, technical skills and backgrounds but a common JS theme - Gulp might well be the best option.

Also, when I inspected the makefile, though I find it quite easy to read, I would have some reservations converting it directly to NPM scripts the way I have done so to date. But again, that might be my own ignorance speaking as I only made the transition fairly recently. Yes, when I think of make I do tend to consider C environments and compiler commands for files. The fact that the makefile has so many .PHONY statements suggests it is sort of being used perhaps a little out of context?

[buzzdecafe]

I'm fine with a Makefile, but I run kubuntu. I did not think windows support would be an issue; I really assumed only ramda core people would be interested in working on the docs site. That assumption is evidently unfounded.

That said, put me down as another vote for vanilla node scripts. I did a fair amount of work disentangling this stuff from grunt, I really do not want to go back in that direction.

[CrossEye]

@branneman, @arcseldon:

And I am not alone in this thinking either - Why I left Grunt and Gulp for NPM Scripts.

Sure, but writing an article about something doesn't make it true :) I disagree wholeheartedly with that article. I think the real problem that the article should address, is overuse and reliance on (bad) gulp plugins.

Of course saying it does not make it so, but it is not just one article. There's a whole chorus of them. The first one I saw was Keith Cirkel's Why We Should Stop Using Grunt. And I really like this point from it:

None of these build tools work without plugins. Just found an awesome new tool which will revolutionise your build process? Great! Now just wait for someone to write a wrapper for Grunt/Gulp/Brocolli, or write it yourself. Rather than just learning the command line for the tool you want to use, you now have to learn its programatic API, plus the API for your build tool. Sindre Sorhus can only write so many of these tools for you.

It got me to try this, and for a number of small projects, it's been perfect. But I haven't had any large projects to try it on, so I don't know how it will scale. I do know that Grunt/Gulp files are getting as bad as the old Ant files I used to maintain, though, and that bothers me.

The only thing that scares me about the NPM scripts is how easy it would be to go the Make direction and create scripts that won't work cross-platform. I no longer can easily test in Windows, but I still would really like for Ramda to build in Windows. If we can be fairly certain that we would have that, then I'd be all for retiring Make in favor of NPM. If we cannot, then I would reluctantly support Gulp.

[CrossEye]

The question is, are the core contributors open to a PR replacing Make with Gulp?
//cc @buzzdecafe @CrossEye @davidchambers

While those three have the longest history, just for the record, the current core team consists of

• @asaf-romano
• @buzzdecafe
• @CrossEye
• @davidchambers
• @paldepind
• @raine
• @scott-christopher
• @TheLudd

[arcseldon]

@CrossEye - your experiences closely match my own.

I do know that Grunt/Gulp files are getting as bad as the old Ant files I used to maintain, though, and that bothers me

That is interesting. Again, I have similar sentiments too. And Webpack is starting to remind me of Maven too... aghh..

Re. above - Sorry, perhaps I should not be posting here as not member of core team?

[buzzdecafe]

@arcseldon please keep posting. your contributions are welcome

[branneman]

None of these build tools work without plugins. Just found an awesome new tool which will revolutionise your build process? Great! Now just wait for someone to write a wrapper for Grunt/Gulp/Brocolli, or write it yourself.

This is just plain wrong with the existence of tools like vinyl-transform. I have a lot of Gulp tasks where I'm not using a gulp- prefixed plugin.

There's too much misinformation on this subject, and those articles are not helping. Bloat is not the fault of Grunt/Gulp, you're using them wrong if this is your problem.

but I still would really like for Ramda to build in Windows

Windows support does not come or go with the tool, but how you use it. They all support it, if used correctly.

Don't get me wrong, I like npm scripts as well, they have their use case for smaller tasks and build jobs. But don't be fooled, it's just as easy to create an unmaintainable (or single platform) mess as it is with Make or Gulp.

[CrossEye]

But don't be fooled, it's just as easy to create an unmaintainable (or single platform) mess as it is with Make or Gulp.

I am actually concerned that it's easier to create a single platform mess with NPM scripts than I would with Gulp. But by experience with Grunt and Gulp, and with Ant before them, always ended in bloat. I haven't yet had that experience with NPM scripts. Perhaps I simply haven't been doing it long enough. 😄

[CrossEye]

@arcseldon:

Re. above - Sorry, perhaps I should not be posting here as not member of core team?

No, @branneman asked for comments from the core team, but specifically callled out only three members. I was simply pointing out that the core team is larger than that.

[CrossEye]

The question is, are the core contributors open to a PR replacing Make with Gulp?

So my answer is, I'd be more interested in one using NPM scripts. But I would still prefer Gulp over Make for the cross-platform support. I would expect that to raise many more objections, though.

[branneman]

@branneman asked for comments from the core team, but specifically callled out only three members

Sorry :( Where can I find a definite list of which contributors are 'core'? I simply did not know.

[buzzdecafe]

I'd be more interested in one using NPM scripts.

Me too, as noted above.

@branneman I appreciate your gallant fight for a node build tool against dogged (if passive) resistance. But for me, you have not yet made a persuasive case for Gulp (Grunt is out of the question for me) over plain vanilla scripts. The disadvantage of adding a build tool is another dependency. That might be ok, if the benefits of that extra layer were manifest.

As far as I can tell, your only arguments against roll-yer-own are:

• NIH syndrome: Maybe; but we are not reinventing the wheel. We're not rying to build a general-purpose build tool. We're writing scripts specifically to build this site. To my mind, it's not clear what advantage gulp provides here.
• Less declarative code: This seems easy to overcome -- that's what we do 😄

[buzzdecafe]

@branneman

Where can I find a definite list of which contributors are 'core'?

https://github.com/orgs/ramda/people

[arcseldon]

@CrossEye - thanks for clarification. Appreciate it.

Re.

Grunt is out of the question for me

Laughing out loud. Yep, you're another one that got the T-Shirt.

Personal opinion only, I could be persuaded with a Gulp file, but as you point out, really we are talking about a small script and any solution so long as cross-platform might be good enough to get started.

[buzzdecafe]

Grunt is out of the question for me

Laughing out loud. Yep, you're another one that got the T-Shirt.

@arcseldon yes, I'm remembering the pain of extracting ramda from the clutches of grunt. we replaced that with a home-grown build script and have never looked back. That experience is certainly informing my opinion here.

[davidchambers]

I have my preferences, but the most important thing is for the people most excited about contributing to this project to be happy with the accompanying tools.

My vision is of something dramatically simpler. I'm working on a website for Sanctuary which is built by running one script – no need for any of the tools discussed in this thread. Once it is ready, I hope it will inspire us to simplify this project in similar ways.

[CrossEye]

@branneman asked for comments from the core team, but specifically callled out only three members
Sorry :(

Oh, it wasn't a big deal. I just wanted to give credit to others, especially to @asaf-romano, who joined the core team just this week.

[CrossEye]

@davidchambers:

My vision is of something dramatically simpler. I'm working on a website for Sanctuary which is built by running one script – no need for any of the tools discussed in this thread. Once it is ready, I hope it will inspire us to simplify this project in similar ways.

I really look forward to seeing that!

[MattMS]

Quite a surprise seeing how much discussion my little prompt spawned. 😄

I'm on Windows and I'm keen to help with the site, so I'm happy to act as the canary in the Windows mine.

I was planning to use npm scripts for the single commands (Gitbook, JSDoc, Less) but the download and build pages would likely require a simple script.
Maybe I could write that up and people can assess if a Gulp task would be simpler?

I just wanted to check people were happy with me going ahead and get confirmation that task list was complete.

[buzzdecafe]

Quite a surprise seeing how much discussion my little prompt spawned.

We all like to talk. A lot.

Maybe I could write that up and people can assess if a Gulp task would be simpler?

SGTM

[CrossEye]

Maybe I could write that up and people can assess if a Gulp task would be simpler?
SGTM

And to me.

[MattMS]

Cool. Less is easy but I'm a bit stuck with the GitBook and JSDoc commands.

• Less is fine: lessc ./less/ramda.less ./style.css
• GitBook makes a pdf: gitbook build manual
• JSDoc throws errors: jsdoc --destination ./tmp --template ./jsdoc/templates ./docs/dist/ramda.js

I thought it would be simplest to split the commands into 2 steps.
So first it does the compilation to a common area and then copies the files into the version folders.

[TheLudd]

The question is, are the core contributors open to a PR replacing Make with Gulp?

Late to the party here. I will always recommend npm scripts as the build tool but have no strong opinion on what ramda uses.

[davidchambers]

My vision is of something dramatically simpler. I'm working on a website for Sanctuary which is built by running one script – no need for any of the tools discussed in this thread. Once it is ready, I hope it will inspire us to simplify this project in similar ways.

I really look forward to seeing that!

Have a look at sanctuary-js/sanctuary-site#2. :)

Note that I do use Make because I appreciate a dependency graph, but one can run scripts/generate node_modules/sanctuary directly if desired.