very hard to make components consumable in node.js

[matthewmueller]

This seems like a sticky issue, and maybe npm and component are incompatible, but any component with other dependencies is nearly impossible to consume as an npm module.

This is because we have, say component/each within component/collection. Well if we wanted to make component/collection a node module, we'd need require('each') to be require('each-component')

This makes it very hard to take advantage of creating modules that work on both client-side and server-side, unless they're trivial (no dep) components.

[yields]

+1

i think we can solve that easily:

take uid as an example, you can require it client side with MatthewMueller-uid (i think ?)
so publishing it to npm with the name MatthewMueller-uid should work?

anyway it's an issue, doing the try catch thing is PITA.

[matthewmueller]

Yah, that's what I was kind of thinking but wasn't sure that was going to work. Instead of publishing each-component, publish as component-each.

[tj]

npm supports user/repo syntax now (sorta), so you would have to have a package.json but otherwise it does work, still not a bridge I'm too concerned about personally, a very large percentage of our components are browser-only, not something i want to get into

[tj]

@matthewmueller we should try using the GH syntax with component/model's package.json deps and see how that goes, it should be pretty seamless

[matthewmueller]

Oh cool, I'll take a look at the repo key.

Yah, lots of components are browser only.. but there's also a bunch of js components that are generic enough to work on both sides. A consistent API without writing two different modules is a huge win.

[tj]

I think with npm it's:

"dependencies": {
  "set": "component/set"
}

not sure how you get the version in there, # maybe

[matthewmueller]

interesting, yah i'll take a look at it now

[tj]

best part of that too is you avoid all the authentication stuff!! double win

[matthewmueller]

sweeeet. ok yah, by specifying deps like that you can completely bypass npm publishing.

Opened two issues on NPM to try and get some things resolved: https://github.com/isaacs/npm/issues/3013 and https://github.com/isaacs/npm/issues/3014.

[matthewmueller]

TL;DR: package.json and component.json should have the same name.

Okay so I think we should be encouraging people to not publish components to NPM, unless they can get the exact name as the component's name. Here's why:

"dependencies": {
  "set": "component/set"
}

will create node_modules/set-component because the name in package.json is set-component. This becomes incompatible with other js components that would like to consume set both client-side and server-side.

E.g. - node.js's require will be require('set-component'), while component's require will be require('set')

I think this tradeoff is fine. The only thing you currently lose is the ability to specify specific versions when you want to use components in node.js.

[tj]

and the discoverability in npm goes out the window

[matthewmueller]

mmm yah that's true too.

[jhnns]

Why does requiring dependencies work differently than in node?

If dependencies would be installed like

components/
     userA/
          moduleA
     userB/
          moduleA

and always required with require("userA/moduleA") it would be easier to achieve a common codebase. Than all I would have to do is to add the components-folder to NODE_PATH (or am I getting it wrong?).

I really like your idea of components and I understand your decision, to make components independent from npm. But I don't understand why this have to be different than in node. It just adds another fragmentation imho.

[vendethiel]

@jhnns everything is first-class with component, which doesn't work well with non-flat directories. I agree I'd like to be able to use require('./templates/item') from user/views/item-view

[tj]

@jhnns it's a little hard to describe actually, but the thing is everything in npm-land (or component-land) is nice and modular, until it gets to your application. Then suddenly everyone just uses a weird awkward tree of modules and does all these weird relative requires. To me why does the true modularity stop at the app level? What's so special or different about the app? An app is just a series of private modules, that should otherwise be identical to those in npm IMO, first class.

HOWEVER I do agree that this ups the learning curve, I need to think on it some more and see if we can compromise to facilitate both styles

[jhnns]

I totally agree. That's what I like about component. This also makes it easy to publish certain parts of your application open source without exposing app-specific stuff - because you write a modular app from the start.

It's also a good idea to decouple component from node in order to open it to other communities. But decoupling shouldn't mean separation. One great advantage of node is that it uses the same language as browsers. And I want to re-use certain components in both environments.

Whether component will be the place for browser-only or common JavaScript modules - I think it's a good idea to adopt some conventions from the node community in order to improve interoperability.

However, I do like the <user> / <project> convention 😺

[vendethiel]

no I agree that your features things should be in components however in my app I have a module register()ed for each controller/action, how am I to structure that ?

[tj]

@Nami-Doc not sure what you mean, can you gist an example?

[vendethiel]

well for example I have to create a form validator for registration, I'll have a module for form validations, required by my module controllers/Account/create, itself required in require('MyApp');

[tj]

why not have (something like):

./account
  ./validator.js
  ./controller.js
  ./view.js
  ./account.css
  ./component.json

etc, that's sort of what I mean by why do we build our applications rails-style, while we build our communities modular

[vendethiel]

I currently have http://i.imgur.com/vBJuQot.png
so yeah the controllers/Account/create will require controllers/Account/validator etc
it's autorequired from the php action I don't want to mess things

[pygy]

Another solution to this would be to add a component build phase to the npm install process, ie fetch all the deps with component, and build a standalone library.

You'd add a "component" dependency to package.json, and list component deps only in component.json.

Redundancy is the norm on the server side, anyway.

[matthewmueller]

anyone else still stuck? fwiw, I'm still running into this again and again.

I think the appropriate solution would be for NPM to allow require('emitter'), where the package.json is:

"dependencies" : {
  "emitter" : "component/emitter"
}

This would be regardless of the emitter's package.json, which might look like this:

{
  "name : "emitter-component"
}

This allows you to consume node modules in two different ways:

npm central-registry style:

"dependencies" :  {
  "emitter-component" : "*"
}

var emitter = require('emitter-component');

github style:

"dependencies" :  {
  "emitter" : "component/emitter"
}

var emitter = require('emitter');

ya dig? here's the corresponding issue in npm: https://github.com/isaacs/npm/issues/3144.

[yields]

@matthewmueller +1 !

[clintwood]

@matthewmueller +1 here too... this is killing me! I'm wrapping to sync the names and that is not cool!

[stereosteve]

One idea that would not require a change to NPM would be to adopt the github naming convention when publishing components to NPM, and adding support for github naming convention in component (using aliases).

For instance, when publishing emitter to NPM:

{
  "name : "component-emitter"
}

Then add extra aliases to component so that you can require things by their component name or full github name. For instance, make it so that both of these would work in component land:

var emitter = require('emitter');
// or
var emitter = require('component-emitter');

By using the later you would have code that would work in both node and the browser without env checks.
Of course, this sucks in the long run, like if you want to use a different fork

[eldargab]

+1

We should probably vote in npm issue as well.

[isaacs]

Really? ALL names are literally taken? I think that's pretty obviously not true.

[yields]

I think what @Nami-Doc meant to say was "finding descriptive names is almost impossible..",
i don't think someone could argue with that.

[isaacs]

Well, actually, that's exactly what I'm arguing with. So far 4766 brand new packages have been created with unique names, just this year. 742 of those were in the last week. Almost 100 of them JUST TODAY. Are you saying that not even ONE of these is "descriptive"?

https://isaacs.iriscouch.com/registry/_design/app/_view/analytics?group_level=1&startkey=[%22created%22,2013,3]&endkey=[%22created%22,2013,3,{}]&group=true

Frankly, if you're having trouble coming up with a reasonable name for the thing you're building, you're probably building the wrong thing. I've written almost 150 modules published to npm, and a bunch that haven't been published. Finding an un-taken name is not a real problem, and when the best name is taken, even better! You've found someone else solving the same problem as you, so you should work with them.

[yields]

Are you saying that not even ONE of these is "descriptive"

Never said that, i said almost impossible, so the question should be:
Are you saying that most of these are descriptive ? surely the question cannot be yes...

{"key":["created",2013,3,1,"xcas"],"value":1},
{"key":["created",2013,3,1,"y"],"value":1},
{"key":["created",2013,3,1,"ctx"],"value":1}

btw i'm happy with npm as it is, but it would be great if you think about this feature, if you decide to have it (or a portion of it), I will be happy to give back to npm and actually implement it... :)

[isaacs]

Never said that, i said almost impossible, so the question should be:

So, if you're going to split hairs, then there's no such thing as "almost impossible" because that would mean it's "possible", and thus, not impossible.

My point is that, if you're having a hard time picking a name, and that's really your biggest problem, you're probably solving the wrong problems, or engaging in NIH.

My complaint about your criticism is that it's hyperbole. I have no problems finding new names. Neither does @substack, or @Raynos, or @dominictarr, or many other prolific node module authors. Are you saying that "emitter" is a more descriptive name than "component-emitter" for the emitter module that is component-specific?

The fact is that @visionmedia has made design choices in component that make it less compatible with npm. That's fine. But that's no reason to rewrite npm so that you can call things different names. Tell TJ to rewrite component so that it uses the name from package.json, if it really bugs you so much.

it would be great if you think about this feature, if you decide to have it (or a portion of it), I will be happy to give back to npm and actually implement it... :)

I explained the requirements over at https://github.com/isaacs/npm/pull/3205#issuecomment-14654258

[yields]

Looking at "requirements", it seems that this feature isn't going to be implemented, ever.

At this point, we may decide that this feature is, in fact, too horrible to even consider, and it will die a bitter and tragic death, once and for all

We should look into other solutions.

[Raynos]

@isaacs actually naming things is super hard.

Not just npm modules. Functions and variables too. I waste like 50% of my time figuring out what to name the damn things.

Actually having a proper namespace in npm is god send. It's something enforcing uniqueness and consistency. Do you appreciate how awesome it is to have require("foo") and npm docs foo mean the same thing everywhere.

[dominictarr]

Don't say "naming things is hard", that is just giving yourself an excuse to be lazy about naming things well.
Instead, say "naming well is a challenge", that implies that while it's difficult, it's possible, and worthwhile.

[eldargab]

May be just not to put things to npm? Or if you want to make your module easier to discover just publish a dummy version:

{
  "name": "username-foo",
  "dependencies": {
    "foo": "username/foo"
  }
}

module.exports = require('foo')

That's for names which are essentially taken. There are few such. They all belong to very simple elementary tasks. For example UI elements - tip, dropdown, menu, etc. Each app, developer or organization might have it's own idea about what such names should mean exactly. Saying "be creative and pick another" just doesn't make sense. Prefixing with username-? Not too bad actually, but in the component word the choice is already made and it makes sense.

On other hand there are tasks for which there is no one precise word describing what it does or tasks for which we really want only one canonical solution. For such case I agree with isaacs - "go and be creative".

[tj]

The names people have been giving things are pretty ridiculous lately but I don't see how npm can get around that now, I don't care much about things being in both places personally, the only things I have "ported" from npm to component is the odd utility or two. I think creative naming is great for things that actually have a creative aspect, stuff like fleet etc, but simple utilities shouldn't really need to be named emitter-component, events-component-2, uid2, better-assert, uid-util, etc. There are certainly pros and cons to our approach, like @Raynos mentioned it's not like you have to say "use tj's uid" or "use guillermo's uid", but I definitely prefer the freedom this approach gives you. I dont think npm compatibility is a huge deal, the lame try/catch approach is ugly but it works for now, I don't want to "revert" to a non-scoped registry just for that. Using npm's newer github-style support is working pretty well, you just don't get the perks of being in the registry (or the bad parts like managing the ownership etc). npm has been around for a lot longer I dont think it would make much sense for npm to change in that respect unless it was beneficial regardless of components.

[Raynos]

as TJ says. If you treat node and frontend as two disjoint special childs then use npm for node and use component for frontend. For frontend stuff component is nicer.

If your into isomorphic code and want to treat node & frontend as the same JS with minor host environment differences then suck it up, use npm and browserify, if they suck then improve or fork them.

Or if your really crazy you can hack in a npm-component interop thing. Like using a fork of the npm client locally, that could work. Good luck with that!

[medikoo]

It's always though to come up with good original name for your utility, and it's not 5000 names taken that makes that difficult. I have about 20 packages published and about 10 I plan publish soon, and names was never a blocking issue for me (even though few times, name I wanted was already taken).

I also don't see a solution in naming your utils as my-nick/util, it doesn't bring nice recognizable names but instead brings no name feeling. It might be ok if you do stuff for yourself, but if you also care for outer adoption it probably won't help.

I'd say: if your after generic cross-environment packages just use NPM, if you're ok just with browser world, you might be fine with Components. I'm personally totally happy with NPM although 70% of my work is dedicated for client-side.

[tj]

If you're happy with names like enchilada or falafel all the power to you, the point is we have the choice.

[medikoo]

@visionmedia the only "advantage" you have is that you can import namespace/whatever with require('whatever'), that's all. I don't see any big opportunity in that.

Externally your package remains namespace/whatever, from that perspective there's no single advantage over NPM. Some NPM users used namespace based naming method ("namespace-whatever") even before Component existed.

[tj]

If you're happy with npm, use npm, there's nothing "wrong" with npm really, just things we feel could be different so we're going that route, there's certainly no reason to ditch it if you have no issues. For us this means many things including:

• an obvious / canonical location for browser related libs
• knowing these libs are designed explicitly for the browser (aka they actually work, no .forEach shims etc)
• better discoverability
• no need for redundant ownership management that github already provides us
• no cache headaches
• no name collisions
• nicer search

etc.

[matthewmueller]

Well I think this is a big shame. It's all javascript. Perhaps right now it's an odd utility or two, but by making this easy and encouraging common libs, we open the doors to a bunch of new possibilities. For example, component/s3 and learnboost/knox could pretty easily be merged into a lib that works on both sides. From there you get a single repo for developers to come to, report bugs, create pull requests. Right now try-catch is fine for a single utility, but when you need a bunch of utilities across a bunch of files, it gets messy quickly.

I do think the component's "de-centralized approach" is a more scalable approach, that's why I was hoping to get support from npm, even though npm is older, more established, and working fine. Perhaps, we were hyperbolizing that it's hard to find names in NPM. Yes, you can find new names for things. But as TJ said, it's the utilities that get annoying. The problem is that the utilities are exactly what you want working on both sides.

[Raynos]

how is github any more "de-centralized" then npm. npm can hook into any registry you want, it's just a couch client.

[matthewmueller]

all i mean is namespacing (component's raynos/graphics, visionmedia/graphics vs. npm's graphics)

[dominictarr]

A browser is not it's own thing that needs it's own package manager - it's just another VM. A VM that you don't get root access to, and has a few weird handicaps, but like VMs in a datacenter, it's just a sandbox that you can safely install arbitrary programs to over the network. Instead of writing your program to fit into the differences between the server and the client - what about designing your program to take advantage of the similarities?

After all, that is one of the big wins of using js on the server and js on the client.

[matthewmueller]

@dominictarr so the programs we would write would take advantage of the similarities, just abstract out the differences and provide a common api.

or are you saying there's no need for a package manager on the client side?

[Raynos]

@matthewmueller prefix your modules with your name. easy enough. I don't have this naming problem. I authored 206 modules.

dominictarr means the differences in host environments between frontend and backend are trivial. Write isomorphic code.

[matthewmueller]

@Raynos I was taking a look at some of your repos. Are you even publishing to npm or just doing:

"dependencies" : {
  "graphics" : "Raynos/graphics"
}

or are you saying I should just call my modules:

"name" : "matthewmueller-graphics"

---

This whole issue has gotten lengthy (perhaps spammy for some), but the approach we could advocate for in component is to not even publish to npm, just have in the docs:

For npm:

npm install raynos/graphics

For component:

component install raynos/graphics

Downsides mentioned earlier: discoverability

I'm not sure how many people run npm search graphics and just install it right away without going to the github page anyway. I think most discoverability comes through stars and visibility of repo on github. Could be wrong though. I guess repos wouldn't even show up if you did npm search graphics, might be a problem.

[Raynos]

@matthewmueller graphics is an experiment. It's not ready for npm yet. When it's ready I will publish it to npm under the name graphics.

I refuse to use git for downloading and installing dependencies. It's a nightmare because you can't do npm i [email protected]

@matthewmueller discoverability is a seperate and difficult problem. npm is for VERSION MANAGEMENT.

[dominictarr]

@matthewmueller I'm saying there should just be one package manager that works for both, either use npm on the client side, or component on the server-side.

[Raynos]

Aw man. That's a great idea. Use component on the server side!

[tj]

@matthewmueller s3 on the client is completely different than on the client, that's the case with a lot of this code, we literally use maybe 5 things from npm.

The solution is pretty simple, if you don't want to use it, don't, if you don't like the npm experience, use something else like components. I don't want the ambiguity of poorly written server code that can be hacked into working on the browser. Using one for the client and one for the server also helps make it extremely clear which dependencies belong where, vs going into your code to see what you happen to require() on the client.

The isomorphic argument is fine, no reason you can't still do that, we just publish to both, one with a stupid name, one with the name we actually want. I would actually prefer if a lot of our server code was less like our client code, node could be much better without all the callbacks, but it is what it is, we do end up sharing a lot of utilities and our database related code but not using npm doesn't get in the way of that at all for us.

[eldargab]

I created a module which allows to require components from node programs as well as to easily share them with npm users.

https://github.com/eldargab/component-as-module